Selecting two-dimensional imagery data for display within a three-dimensional model

ABSTRACT

Systems and methods for generating three-dimensional models with correlated three-dimensional and two dimensional imagery data are provided. In particular, imagery data can be captured in two dimensions and three dimensions. Imagery data can be transformed into models. Two-dimensional data and three-dimensional data can be correlated within models. Two-dimensional data can be selected for display within a three-dimensional model. Modifications can be made to the three-dimensional model and can be displayed within a three-dimensional model or within two-dimensional data. Models can transition between two dimensional imagery data and three dimensional imagery data.

TECHNICAL FIELD

This disclosure generally relates to systems and methods for selectingtwo-dimensional data for display within a three-dimensional model, andmore particularly to transitioning between rendering three-dimensionalimagery data of a three-dimensional model and rendering two-dimensionalimagery data associated with the three-dimensional model.

BACKGROUND

Interactive, first-person three-dimensional immersive environments arebecoming increasingly popular. In these environments, a user is able tonavigate through a virtual space. Examples of these environments includefirst person video games and tools for visualizing three-dimensionalmodels of terrain. Aerial navigation tools allow users to virtuallyexplore urban areas in three dimensions from an aerial point of view.Panoramic navigation tools (e.g., street views) allow users to viewmultiple 360-degree panoramas of an environment and to navigate betweenthese multiple panoramas with a visually blended interpolation.

Three-dimensional immersive environments can be generated based onmanual user input or based on automatic three dimensional reconstructionsystems. Manual generation of these environments is often costly andtime consuming. Three-dimensional immersive environments can also haveerrors or holes. With these developments, there is a consequential needto view and navigate these three-dimensional models on an array ofcomputing devices.

SUMMARY

The following presents a simplified summary of the specification inorder to provide a basic understanding of some aspects of thespecification. This summary is not an extensive overview of thespecification. It is intended to neither identify key or criticalelements of the specification nor delineate the scope of any particularembodiments of the specification, or any scope of the claims. Itspurpose is to present some concepts of the specification in a simplifiedform as a prelude to the more detailed description that is presented inthis disclosure.

Systems, methods, and apparatuses are disclosed herein to facilitategenerating correlated models comprising three-dimensional (3D)reconstructions of captured content and two-dimensional (2D) capturedcontent. A 3D model can be generated based on sensor data (e.g., 3Dsensors), 2D image data, and the like. The 3D model can comprisepositional data for rendering 3D images. 2D images can be correlated andoriented with the 3D model.

Models can be rendered via an interface. Rendering of the models cantransition between rendering 3D data, 2D data, or a combination of 3Dand 2D data. 2D data can be selected for transitioning to based onselection criteria. Selection criteria can comprise a point of viewassociated with a rendered model, analysis of the 2D data or 3D data, ornavigational data. Notifications can be generated to signify whetherrendering can alternate between rendering 3D data, 2D data, or acombination of 3D and 2D data. Alternating between rendering 3D data, 2Ddata, or a combination of 3D and 2D data can include smoothtransitioning, panning, snap transitions, and the like.

A set of user tools can facilitate manipulation of 3D and 2D models.User tools can allow for adding user generated content, removingportions of captured content, manipulating a rendering, generatingsummaries, and the like.

The following description and the drawings set forth certainillustrative aspects of the specification. These aspects are indicative,however, of but a few of the various ways in which the principles of thespecification may be employed. Other advantages and novel features ofthe specification will become apparent from the following detaileddescription of the specification when considered in conjunction with thedrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Numerous aspects, embodiments, objects and advantages of the presentinvention will be apparent upon consideration of the following detaileddescription, taken in conjunction with the accompanying drawings, inwhich like reference characters refer to like parts throughout, and inwhich:

FIG. 1 illustrates a high-level block diagram of an example system thatcan alternate rendering 3D images of a 3D model and rendering 2D imagerydata in accordance with certain embodiments of this disclosure;

FIG. 2 illustrates a graphical depiction of 2D panoramic images within a3D model in accordance with certain embodiments of this disclosure;

FIG. 3 illustrates a high-level block diagram of an example system thatcan capture media content and generate models in accordance with certainembodiments of this disclosure;

FIG. 4 illustrates a high-level block diagram of an example system thatcan capture content in accordance with embodiments of this disclosure;

FIG. 5 illustrates a high-level block diagram of an example system thatcan facilitate navigation of models in accordance with certainembodiments of this disclosure;

FIG. 6 illustrates a high-level block diagram of an example system thatcan facilitate navigation, manipulation, and control of a model inaccordance with certain embodiments of this disclosure;

FIG. 7 illustrates a high-level block diagram of an example system thatcan facilitate display of a model in accordance with certain embodimentsof this disclosure;

FIG. 8 illustrates an example methodology that can provide forgenerating models and transitioning between 3D imagery data and 2Dimagery data in accordance with certain embodiments of this disclosure;

FIG. 9 illustrates an example methodology that can provide for selecting2D images for transitions based on navigation positions and field ofview in accordance with certain embodiments of this disclosure;

FIG. 10 illustrates an example methodology that can provide fortransitioning from a 3D model to a 2D model in a 3D modeling system inaccordance with certain embodiments of this disclosure;

FIG. 11 illustrates an example methodology that can provide fortransitioning between a first mode and a second mode in accordance withcertain embodiments of this disclosure;

FIG. 12 illustrates an example methodology that can provide forenhancing a field of view in accordance with certain embodiments of thisdisclosure;

FIG. 13 illustrates an example schematic block diagram for a computingenvironment in accordance with certain embodiments of this disclosure;and

FIG. 14 illustrates an example block diagram of a computer operable toexecute certain embodiments of this disclosure.

DETAILED DESCRIPTION

Various aspects or features of this disclosure are described withreference to the drawings, wherein like reference numerals are used torefer to like elements throughout. In this specification, numerousspecific details are set forth in order to provide a thoroughunderstanding of this disclosure. It should be understood, however, thatcertain aspects of disclosure may be practiced without these specificdetails, or with other methods, components, materials, etc. In otherinstances, well-known structures and devices are shown in block diagramform to facilitate describing this disclosure.

Terms such as “user equipment,” “user equipment device,” “mobiledevice,” “user device,” “handset,” or terms representing similarterminology can refer to a device utilized by a subscriber or user toreceive data, convey data, control, voice, video, sound, models, gaming,and the like. The foregoing terms are utilized interchangeably hereinand with reference to the related drawings.

Furthermore, the terms “user,” “subscriber,” “customer,” “consumer,”“end user,” and the like are employed interchangeably throughout, unlesscontext warrants particular distinctions among the terms. It should beappreciated that such terms can refer to human entities, human entitiesrepresented by user accounts, or automated components supported throughartificial intelligence (e.g., a capacity to make inference based oncomplex mathematical formalisms), which can provide simulated vision,sound recognition and so forth.

Digital 3D models can be generated based on 2D sensory data, sensorydata in combination with raw 2D data, computer generated positionaldata, and the like. In an aspect, data used to generate 3D models can becollected from scans (e.g., utilizing sensors) of real-world scenes,spaces (e.g., houses, office spaces, outdoor spaces, etc.), objects(e.g., furniture, decorations, goods, etc.), and the like. Data can alsobe generated based on computer implemented 3D modeling systems.

3D models can comprise data representing positions, geometric shapes,curved surfaces, and the like. For example, a 3D model can comprise acollection of points represented by 3D coordinates, such as points in a3D Euclidean space. The collection of points can be associated with eachother (e.g., connected) by geometric entities. For example, a meshcomprising a series of triangles, lines, curved surfaces (e.g.,non-uniform rational basis splines (“NURBS”)), quads, n-grams, or othergeometric shapes can connect the collection of points. In an aspect,portions of the mesh can comprise image data describing texture, color,intensity, and the like. In embodiments, captured 2D images (or portionsthereof) can be associated with portions of the mesh.

It is noted that the terms “3D model,” “3D object,” “3D display,” “3Dreconstruction,” “3D rendering,” “3D construct,” and the like areemployed interchangeably throughout, unless context warrants particulardistinctions among the terms. It should be appreciated that such termscan refer to data representing an object, space, scene, and the like inthree dimensions, which may or may not be displayed on an interface. Inan aspect, a computing device, such as a graphic processing unit (GPU)can generate, based on the data, performable/viewable content in threedimensions. The terms “3D data,” “3D imagery data,” and like areemployed interchangeably throughout, unless context warrants particulardistinctions among the terms and can refer to data utilized to generatea 3D model, data describing a 3D model, data describing perspectives orpoints of view of a 3D model, capture data (e.g., sensory data, images,etc.), meta-data associated with a 3D model, and the like.

In another aspect, terms such as “navigational position,” “currentposition,” “user position,” and the like are employed interchangeablythroughout, unless context warrants particular distinctions among theterms. It should be appreciated that such terms can refer to datarepresenting a position in a 3D model during user navigation and thelike.

Embodiments described here can reference a model in a particular mode orview, such as a walking mode, orbital mode, floor plan mode, 3D mode, 2Dmode, or the like. However, it is appreciated that each mode cancomprise a distinct model. Accordingly, a mode can be defined as adistinct model or a model in a particular mode with determined availablefeatures.

It is noted that the terms “2D model,” “2D image(s),” and the like areemployed interchangeably throughout, unless context warrants particulardistinctions among the terms. It should be appreciated that such termscan refer to data representing an object, space, scene, and the like intwo dimensions, which may or may not be displayed on an interface. Theterms “2D data,” “2D imagery data,” and like are employedinterchangeably throughout, unless context warrants particulardistinctions among the terms and can refer to data describing a 2D image(e.g., meta-data), capture data associated with a 2D image, a 2D image,a representation of a 2D image, and the like. In an aspect, a computingdevice, such as a GPU, can generate, based on the data,performable/viewable content in two dimensions.

In another aspect, 2D models can be generated based on captured imagedata, 3D imagery data, and the like. In embodiments, a 2D model canrefer to a 2D representation of a 3D model, real-world scene, 3D object,or other 3D construct. As an example, a 2D model can comprise a 2Dimage, a set of 2D images, a panoramic 2D image, a set of panoramic 2Dimages, 2D data wrapped onto geometries, or other various 2Drepresentations of 3D models. It is noted that a 2D model can comprise aset of navigation controls.

A 3D modeling system can generate a rendering of a 3D model. In anaspect, the 3D model can be rendered in one or more modes as describedin more detail below. For example, a 3D model can be rendered in awalking mode, orbital mode, flying mode, floor plan mode, and the like.In an aspect, a user can provide input to a 3D modeling system and the3D modeling system can facilitate navigation of the 3D modeling system.As used herein, navigation of a 3D modeling system can include alteringa field of vision, as described in more detail below. For example, afield of vision can rotate about a viewpoint (e.g. an axis or pivotpoint), alternate between viewpoints, enhance a region of a model, altera size of a region of a model (e.g., “zoom in,” or “zoom out,” etc.),and the like.

In accordance with one or more embodiments described in this disclosure,a 3D imaging system can generate a 3D model and select 2D images fordisplay. A construction component can generate a 3D model, such as amodel based on 3D imagery data comprising, for example, image data incombination with, point cloud data, 3D mesh data, volumetric renderingdata, surfel cloud data, cartoon rendering data, and the like. It isnoted that a 3D model can comprise 2D images (color, textures, etc.) andpositional data. As used herein, 2D imagery data can comprise raw 2Dimage data, flat image data, field of view data, or viewpoint data thatrepresents a viewpoint (e.g., capture position) for 2D images.

A selection component can facilitate selecting the 2D imagery data fordisplay. For example, the selection component can select a 2D image fordisplay based on selection criteria. In another aspect, the selectioncomponent can select, based on 3D imagery data and associated viewpointsof a 3D model (e.g., such as a viewpoint of the 3D model or a selectedposition of the 3D model), 2D imagery for display in a 2D mode. Theselection component can pre-select 2D images based on associatedviewpoints and 3D position data or can select 2D images at a run time. Adisplay control component can render or manipulate a 3D model. Thedisplay control component can facilitate navigation in or about a modeland altering modeling modes, dimensions, and the like. In anotheraspect, the display control component can facilitate a transitionbetween rendering a 3D model and rendering 2D images

In accordance with other embodiments described in this disclosure, a 3Dmodeling method is described herein. The method can include generating a3D model based on 2D imagery data and 3D imagery data. For example,captured 2D images can be associated with coordinates of a 3D modelbased on image or feature matching techniques such as bundle adjustmentand other photogrammetry techniques, location data, data from additionalsensors such as Inertial Measurement Units (IMUs), and the like. In someembodiments, the method can comprise navigating a 3D model and providing2D images within the 3D model. The method can include selecting 2Dimages for display within a 3D model, in conjunction with a 3D model, orin place of a 3D model. Selecting the 2D images for display can compriseselecting a 2D image based on a navigation position within a 3D model, aviewpoint of a 3D model, user input, and the like.

In another aspect, embodiments can comprise alternating between modelsor modes. For example, a method can render a 3D model in a first modeand can switch to a second mode. In another example, a 3D walkthroughmode can be rendered, such as via an interface, and, in response to atrigger, a 2D panorama or other 2D image can be displayed. In an aspect,the field of view can be correlated during switching between modes. Forexample, a 3D walkthrough mode can be rendered such that a particularfield of view is displayed. Switching to a 2D walkthrough mode cancomprise rendering a 2D model such that the same or similar field ofview is displayed. It is noted that a 2D walkthrough mode can beconstrained by properties of 2D images. For example, in a 2D walkthroughmode, navigation can be limited to navigating between determinedviewpoints (e.g., points corresponding to capture points) and rotatingaround in various axes while remaining stationary on another axis oraxes.

The above-outlined embodiments are now described in more detail withreference to the drawings, wherein like reference numerals are used torefer to like elements throughout. In the following description, forpurposes of explanation, numerous specific details are set forth inorder to provide a thorough understanding of the embodiments. It may beevident, however, that the embodiments can be practiced without thesespecific details. In other instances, well-known structures and devicesare shown in block diagram form in order to facilitate describing theembodiments.

In implementations, the components described herein can perform actionsonline or offline. Online/offline can refer to states identifyingconnectivity between one or more components. In general, “online”indicates a state of connectivity, while “offline” indicates adisconnected state. For example, in an online mode, models can bestreamed from a first device (e.g., server) to a second device (e.g.,user device), such as streaming raw model data or rendered models. Inanother example, in an offline mode, models can be generated andrendered on one device (e.g., user device), such that the device doesnot receive data or instructions from a second device (e.g., server).

While the various components are illustrated as separate components, itis noted that the various components can be comprised of one or moreother components. Further, it is noted that the embodiments can compriseadditional components not shown for sake of brevity. Additionally,various aspects described herein may be performed by one device or twoor more devices in communication with each other. It is noted that whilemedia items are referred to herein, the systems and methods of thisdisclosure can utilize other content items.

Referring now to FIG. 1, a system 100 is depicted. System 100 can select2D imagery data corresponding to a generated 3D model, in accordancewith various embodiments disclosed herein. Embodiments disclosed herein,for example, generate a 3D model, position 2D imagery data based on the3D model, select 2D images to display based on information associationwith the 3D model, render the 3D model, and the like. Such embodimentscan enable additional features in 3D modeling systems, improve usersatisfaction, and provide other benefits that will be apparent herein.

System 100 can primarily include a memory 104 that stores computerexecutable components and a processor 102 that executes computerexecutable components stored in the memory 104. It is to be appreciatedthat the system 100 can be used in connection with implementing one ormore of the systems or components shown and described in connection withother figures disclosed herein. It is noted that all or some aspects ofsystem 100 can be comprised in larger systems such as servers, computingdevices, smart phones, tablets, personal computers, cameras, and thelike. As depicted, system 100 can include communicably coupledcomponents comprising an imagery component 108 (which can generate 3Dmodels and may select appropriate 2D data to show based on informationassociated with the 3D model). Imagery component 108 can primarilycomprise construction component 110 (which can generate 3D models and 2Dmodels) and selection component 120 (which can select 2D imagery databased on 3D imagery data and associated viewpoints or on navigationaldata).

System 100 can receive input to facilitate generation of 3D modelscomprising 3D imagery data and 2D imagery data. For example, system 100can receive raw 2D imagery data, sensory data, 3D imagery data, and thelike. In some embodiments, system 100 can receive a fully or partiallygenerated 3D model or 2D model. It is noted that system 100 can receiveinput, for example, from a capturing device, sensors, a memory store(e.g., database, storage medium, cloud storage, memory 104, etc.), userinput, and the like.

Imagery component 108 can facilitate generation of correlated models.Correlated models can comprise 3D models with correlated 2D imagery orvideo content, 2D models with correlated 3D imagery or video content,and the like. In some embodiments, construction component 110 cangenerate a 3D model based on 3D imagery data, such as a set of points, ageometric mesh, color data, and the like. In an aspect, constructioncomponent 110 can generate 2D models based on 2D images, 3D imagerydata, location data, and the like.

In embodiments, construction component 110 can generate 3D models basedon 3D imagery data, capturing data (e.g., camera speed, rotation, cameratype, etc), position data, and the like. In an aspect, constructioncomponent 110 can determine position data based on a global satellitepositioning (GPS) system, triangulation (e.g., access pointtriangulation), gyroscope data, motion sensors, accelerometer (e.g. a3-axis accelerometer), IMU, trilateration system (e.g., GPS or a localarrangement of signal emitters), visual odometry calculation, physicalodometry measurement, augmented reality markers, or any variety of acombination of the aforementioned or other technologies. Physicalmeasurements of absolute scale may also be taken. The time of each 3Dscene capture may also be determined. Rough estimation of cameramovement may be computed using various methods such as optical flow or2D or 3D feature tracking. This information may be used to provideadditional clues about alignment when determining automatic alignment of3D scenes.

In another aspect, construction component 110 can generate a compositeof 3D images or a 3D mesh based on sensory data, image recognition, andthe like. It is noted that embodiments allow for utilization of varioustechniques to generate the 3D model. Likewise, 2D models can begenerated, for example, based on stitching of 2D images, overlaying 2Dimages onto a 3D model, and the like.

Construction component 110 can determine positions of objects, barriers,flat planes, and the like. For example, based on 3D imagery data,construction component 110 can identify barriers, walls, objects (e.g.,counter tops, furniture, etc.), or other features of the 3D imagerydata. In an aspect, objects can be defined as solid objects such thatthey cannot be passed through when rendered (e.g., during navigation,transitioning between modes and the like). Defining objects as solid canfacilitate aspects of navigation of a model by a user interacting withsystem 100. For example, a user can navigate through a 3D model of aninterior living space. The living space can include walls, furniture,and other objects. As a user navigates through the model, they can beprevented from passing through a wall or other object and movement mayalso be constrained according to one or more configurable constraints(e.g., viewpoint kept at a specified height above a surface of the modelor a defined floor). In an aspect, the constraints can be based at leastin part on a mode (e.g., walking mode) or type of a model. It is notedthat, in other embodiments, objects can be defined as not solid objectssuch that objects can be passed through (e.g., during navigation,transitioning between modes and the like).

In embodiments, construction component 110 can determine a set ofviewpoints, rotational axes, and the like. For example, constructioncomponent 110 can determine viewpoints based on camera poses, locationdata (e.g., relative location of one or more capturing devices orcaptured content), and the like. In an aspect, a viewpoint can comprisea viewpoint of a 2D image, a viewpoint of a 3D image or model, and thelike. Viewpoints can contain position, orientation, and/or field of viewinformation.

In embodiments, construction component 110 can correlate 2D imagery dataand 3D imagery data. For example, construction component 110 candetermine that 2D imagery data corresponds to a position associated witha 3D model, such as a coordinate of 3D planar space represented as an(X, Y, Z) coordinate, where X represents a position on a first plane, Yrepresents a position of a second plane, and Z represents a position ofa third plane. The position may also include information aboutorientation (e.g., a quaternion in an (X, Y, Z) coordinate system).Additional data may localize the position of different parts of the 2Dimagery data within the 3D model. It is noted that various other namingconventions or positioning techniques can be utilized, such as defininga position relative to an origin point). In some embodiments,construction component 110 can determine positions of corners of a 2Dimage in a 3D model, or the position, orientation, and field of view ofa camera that captured a 2D image or 2D panoramic image. Suchdeterminations may be used to create 2D imagery data for the 2D image.

Selection component 120 can select 2D imagery data to be associated withpositions or viewpoints associated with 3D imagery data or to beassociated with navigational positions. In another aspect, selectioncomponent 120 can generate sets of 2D imagery data and can associate the2D imagery data with respective 3D imagery data and associate viewpointsor navigational positions. In embodiments, selection can occur duringnavigation of a model, during rendering of a model, prior to rendering amodel, prior to navigation of a model, and the like. For example,selection component 120 can select 2D imagery data based on a currentnavigation position (e.g., position of a user) during a live navigationprocess. In another example, selection component 120 can select 2Dimagery data for a set of possible navigation positions in the 3D modelprior to navigation of a model to generate a set of selected 2D imagesrespectively associated with the set of possible positions in the 3Dmodel, such as in a predetermined or precomputed fashion. In an aspect,selection component 120 can select 2D images from 2D imagery datautilized to construct (e.g., via construction component 110) a 3D model,2D imagery data capturing during capturing of 3D imagery data, 2Dimagery data uploaded by a user, or 2D imagery data otherwise receivedby selection component 120.

Selection component 120 can select appropriate 2D images based on aselection criterion or selection criteria. Selection criteria cancomprise a score of a content match, user input, an image qualitymetric, capture position data (e.g., position of a capturing device),angle of view, user input, metadata, and the like. For example,selection component 120 can compare 2D imagery data to 3D imagery datato generate a score. In an aspect, a 2D image can comprise a panoramicimage of a particular scene. The panoramic image can be compared to animage of a 3D model where the image of the 3D model corresponds to thepanoramic rendering generated from a particular viewpoint within the 3Dmodel. In another aspect, selection component 120 can compare 2D imagesto select a 2D image as appropriate. For example, given a position orviewpoint in a 3D model, selection component 120 can select one or moreimages from a set of 2D images (e.g., a set of panoramic 2D images)associated with the position or viewpoint in the model. One or more 2Dimages can be selected as a “best-fit” image, a highest quality image,an image having a shortest distance (navigational or actual distance),or the like. Best-fit images can relate to an image selected based on aselection criterion that attempts to select an image to maximize one ormore metrics. The selected 2D image of the set of images can be storedfor later use or displayed upon selection.

In an embodiment, selection component 120 can receive input indicating auser's desire to utilize an image. For example, a user can provideinput, via an interface, that represents a user's selection of a desired2D image. In another example, a user can provide input indicating theuser's desire to move or reposition a best-fit 2D image for viewing.Repositioning an image can comprise selecting a new viewpoint andreplacing a current viewpoint associated with the image with the newviewpoint. One example of a user-selected location is a point of adisplayed 3D model selected by the user indicating a desired userposition for viewing a 2D image. In this and in other embodimentsdescribed herein, a ray may be cast out from a current viewpoint into a3D model to find the location on the 3D model. The nearest 2D datacapture point from that location can be selected. In another embodiment,the nearest 2D data capture point in the same room or the nearest 2Dcapture point that can be navigated to without passing through or arounda wall or other object of a model can be selected.

In another example, a user-selected location can be a position of a 3Dmodel selected by the user indicating a desired position of the modelfor viewing as a 2D image. A fixed area surrounding the selectedposition of the 3D model, or one that is dependent on the currentdistance from the user position to the position, may be used to define adesired area of the 3D model to be viewed.

In some embodiments, a user-selected location can be a region, such as arectangle or area, of a current field of view of the 3D model. Points onthe 3D model at corners of this region may be used to define a desiredvolume of the 3D model to be viewed as a 2D image. In variousembodiments, a 3D object in a 3D model can be selected (e.g., based onuser input). The selected object's position and size may be used todefine a desired area of the 3D model to be viewed as a 2D image and anappropriate 2D image can be selected for display.

In an aspect, an average or median position of the nearest points in the3D model seen within a region (or a statistical sample thereof) orportions of a selected object visible to the user (or a statisticalsample thereof) can be used to define the center of a desired volume tobe viewed. A standard deviation of these points' positions can be usedto define a desired size or field of view. A collection of nearestpoints can be used in the selection of a desired view. It is noted thatan entire current field of view of a 3D model can be characterized inthe same manner as a selected region; thus, the same characteristics canbe computed on the entire field of view even when the user is notselecting a desired location or region.

The above embodiments describe various ways that selection component 120can represent a desired area of a model to be viewed in 3D. These waysinclude, but are not limited to, a desired central position and adesired field of view, a desired central position and a desired size,and a section of the model (whether represented by points, triangles,voxels, a depth map at a specific viewpoint, or another type of spatialdata) that is desired to be visible.

In various embodiments, selection component 120 can select a 2D image(e.g., a 2D panorama), associated orientation, or cropped field of viewfor display based on one or more factors. The selection of the correct2D image and the like can be determined based on analysis of one or morefactors, including, but not limited to the following:

-   -   Minimizing the direct spatial distance between the current user        position in the 3D model or a user-indicated desired user        position in the 3D model to the capture point of the 2D image;    -   Minimizing the user-navigable spatial distance (e.g., in walking        mode) between a current user position in the 3D model or a        user-indicated desired user position in the 3D model to a        capture point associated with 2D image;    -   Minimizing a change in orientation between a current user        orientation and a viewpoint orientation of a 2D image;    -   Maximizing a portion of a desired section of a 3D model that is        visible in a 2D image or a cropped part of a 2D image;    -   Maximizing a portion of a 2D image or cropped part of a 2D image        that is composed of parts from a desired section of the 3D        model;    -   Minimizing a distance between a desired central position within        a 3D model and a central position of a part of the 3D model        viewed by a 2D image;    -   Minimizing a difference between a desired field of view within a        3D model and a field of view of a 2D image or cropped part of a        2D image;    -   Minimizing a difference between a desired size of a        user-selected area of a 3D model and the standard deviation of a        volume seen by a 2D image or cropped part of a 2D image;    -   Maximizing a level of visual detail discernible on pixels in a        2D image or cropped part of a 2D image, it is noted that this        can be limited to only pixels that correspond to points inside a        desired volume or desired section of a 3D model;    -   Whether a user position in a 3D model and a capture point of a        2D image are in the same room of the 3D model; or    -   Minimizing the median on-screen pixel distance between pixels of        a 3D model as seen in a 2D image or cropped part of the 2D image        and the same locations on the 3D model in the user's current or        desired view of the 3D model.

Selection component 120 can combine one or more of the above factorsusing various weightings to produce a score associated with 2D images.In another aspect, a 2D image (or set of images) that has a best scoreor score above a determined threshold can be selected for display. Thescore can be computed for all possible 2D images (as well as optionallya range of possible center points, orientations, and/or fields of view),or various heuristics may be used to reduce the search space of possiblesolutions.

Images associated with a score above a threshold score can be selectedas candidate images. In an aspect, the score can be a weighted scorebased on selection criteria such as the above described factors.Candidate images can be further compared (e.g., based on a morestringent matching process, user input, and the like) to determine abest possible image for display. Once an appropriate candidate image isselected, the image can be displayed or marked for later use. Inadditional or alternative embodiments, a 2D image with the highest scorefor a given viewpoint of the 3D model or given user-selected desiredcriteria can be directly selected by selection component 120.

In one exemplary implementation, selection component 120 canautomatically and dynamically assign a rank or score to portions of 2Dimagery data or 3D imagery data with use of probability models,inference models, artificial intelligence models, and the like. Forexample, selection component 120 can utilize a dynamic Bayesian networksuch as a Hidden Markov Model (HMM) using a Viterbi algorithm. Forexample, a HMM can be configured for a feature (e.g., distinguishableportion of an image) emphasis, scene/composition emphasis, image qualityemphasis (e.g., resolution, sharpness, contrast, color saturation,noise), emphasis based on one or more selection criterions describedherein, or distance emphasis (e.g., navigational or actual distance)such that comparison, scoring according to a metric, or positioningoccur simultaneously or substantially simultaneously.

In another example, selection component 120 can use a variety ofsuitable artificial intelligence (AI)-based schemes as described suprain connection with facilitating various aspects of the herein describedinvention. For example, a process for learning explicitly or implicitlyhow to select 2D imagery data for a given viewpoint, region, or area ofa 3D model can be facilitated via an automatic classification system andprocess. Inferring or learning can employ a probabilistic orstatistical-based analysis to infer an action that is to be executed.For example, a support vector machine (SVM) classifier can be employed.Other learning approaches include Bayesian networks, decision trees, andprobabilistic classification models providing different patterns ofindependence can be employed. Learning as used herein also is inclusiveof statistical regression that is utilized to develop models ofpriority.

As will be readily appreciated from the subject specification, thesubject innovation can employ learning classifiers that are explicitlytrained (e.g., via a generic training data) as well as implicitlytrained (e.g., via observing user behavior, receiving extrinsicinformation) so that the learning classifier is used to automaticallydetermine according to a predetermined criteria which action to take.For example, SVM's can be configured via a learning or training phasewithin a learning classifier constructor and feature selection module. Alearning classifier is a function that maps an input attribute vector,k=(k1, k2, k3, k4, kn), to a confidence that the input belongs to alearning class—that is, f(k)=confidence(class).

In some embodiments, selection component 120 can generate a set of 2Dimages associated with positions associated with 3D imagery data at atime of construction, at a time of capture, or otherwise prior to usernavigation of a 3D model, automatic navigation of a 3D model, orrendering of a 3D model. For example, selection component 120 cangenerate a set of 2D images (and/or associated viewpoints) that havebeen selected and associated with positions of a 3D model at a time whenconstruction of a model is occurring, at a time after construction of amodel, or at various other times. The generated set of 2D images can beassociated with positions of a 3D model and can be utilized during usernavigation or rendering. In an aspect, selection component 120 candetermine a set of positions and orientations to select 2D images orutilize positions and orientations determined during construction of a3D model. Determining the set of positions and orientations can compriseselecting positions and orientations based on a regularly spaced grid,distances between positions and orientations, user input, a qualitymetric of 3D imagery data (e.g., holes, etc.), capture positions, andthe like and a given user position or orientation can be match to anearest-neighbor. In another aspect, selection component 120 can select2D images based on a quality metric of the 2D images. For example,selection component 120 can select a number of best quality images,according to a quality metric, and can associate the images withpositions and orientations of 3D imagery data. In another example,selection component 120 can select a number of images per room,volumetric space, and the like. In some aspects, a pre-generated set canreduce a burden on a processor during rendering or navigation of a 3Dmodel.

In other embodiments, some or all of the 2D images are selected andmatched at or during a time of user navigation and rendering of a 3Dmodel, or based on a triggering event. For example, selection component120 can generate a 2D image or set of 2D images (and/or associatedviewpoints) as candidates for potential display at a time when renderingof a model is occurring, at a time when a user is navigating a model,based on a user navigating to a position of a 3D model, based on a userviewing a field of view (e.g., a user views a particular field of viewfor a determined amount of time), according to user input, or the like.In another example, one or more 2D images can be selected based on anavigational position, based on a field of view, based on user input(e.g., user selecting an option to show all 2D images and/orviewpoints), based on focusing on a volumetric space of a model, or thelike. In an aspect, a pre-selected 2D image or set of 2D images (orassociated viewpoints) utilized for transitions based on positions of a3D model can avoid unnecessary processing during rendering ornavigation. For example, if a user never navigates to a portion of a 3Dmodel, the selection component 120 can forgo selection of 2D images forthat portion of the 3D model.

In another aspect, selection component 120 can determine whether andwhat 2D images should be selected and matched prior to rendering ornavigation. Determining whether and what 2D images to select prior torendering or navigation can be based on a history of use (e.g., by theuser or a set of users). In embodiments, selection component 120 canlearn or identify patterns of use such as navigation patterns of usersand the like. Selection component 120 can utilize identifiable patternsto determine a likelihood of whether a position or area within the 3Dmodel will be associated with a request to alternate between modes, auser navigation, or the like. In another aspect, the identifiablepatterns can indicate a time period associated with the likelihood. Forexample, a user may be more likely to view a particular area of a 3Dmodel during a time period beginning just after rendering a 3D model andlasting a determined amount of time. Accordingly, selection component120 can download and/or select 2D images associated with the area, priorto rendering, such that an initial processing demand is reduced.

In another aspect, system 100 can perform (e.g., via one or morecomponents) various transforms, alterations, or adjustments to imagerydata. For example, a 2D image can be captured in different lighting orwith a different camera compared to capturing of a 3D image. System 100can alter the images to compensate for differences in capturing, such asthrough adjusting of image fields (e.g., hue, intensity, etc), croppingimages, noise reduction, and the like. It is noted that altered imagescan then be compared to improve or alter potential scores. In someembodiments, system 100 can stitch 2D images to form panoramic imagesand can normalize each image that forms the panoramic image.

In various embodiments, selection component 120 can select viewpoints ofa model to match with viewpoints or locations of a disparate model orimage. For example, a 3D model can comprise position data (e.g.,positions in a 3D space) and a 2D image can be associated with a set of2D viewpoints (e.g., points of capture, available points that 2D imagescan be viewed from, etc.). Selection component 120 can associate each ora subset of the 2D viewpoints with a set of positions in the 3D model.Additionally or alternately, selection component may choose a particular2D viewpoint for a given 3D viewpoint. The association can be based onpositional data (e.g., distance), content matching, and the like. Adistance can represent an absolute distance (e.g., based on coordinates)or a navigational distance (e.g., a distance based on what a userwalking on the surface of the model would have to traverse in order tonot pass through objects, walls, etc.). For example, a 3D viewpoint canbe located on an eastern side of a wall and a 2D viewpoint can be on awestern side of the wall. The distance between the 3D viewpoint and the2D viewpoint can be the navigational distance measured from the 3Dviewpoint, around the wall (without passing through the wall), and tothe 2D viewpoint. The selection component 120 may choose an associated2D image for a given 3D viewpoint by minimizing a metric of distance, ormay use this metric of distance as part of a computed score forselecting associations.

In another aspect, construction component 110 can receive 2D imageselection data from selection component 120. The image selection datacan be applied or attached to a model. For example, given a 3D model,construction component 110 can apply selected 2D imagery data to the 3Dmodel. This can include overlaying 2D images on the 3D model, adding 2Dviewpoints (or representations thereof) to the 3D model, generatingtransitions between the 3D model and 2D images, and the like, as isdescribed in more detail below.

In embodiments, construction component 110 can render a 3D model and canfacilitate displaying a field of view of the 3D model. In someembodiments, if a 2D image is available and associated with the aportion of the 3D model, construction component 110 can render the 2Dimage, provide a notification of the availability of the 2D image, ortransition from the 3D model to a 2D image. In other embodiments, anappropriate 2D image can be selected based on a field of view, adistance to a viewpoint (e.g., actual or navigational distance), userinput (e.g., user selecting a particular area or volumetric space), alearning process, and the like. The selected 2D image or arepresentation of the 2D image can then be displayed or can be displayedbased on a triggering event.

In an aspect, the 2D image can be a panoramic image of the living spaceand switching from the 3D model to the 2D image can include switching toa selected viewpoint and at a determined degree of rotation about anaxis of the viewpoint. For example, a user can be viewing north in a 3Dmodel and can decide to switch to a 2D panoramic image. Constructioncomponent 110 can render the 2D panoramic image such that the user willbe viewing north. It is noted that proper alignment and orientation of2D imagery data can allow for appropriate transitions.

Referring now to FIG. 2 with reference to FIG. 1, there illustrated is agraphical depiction of system 200 that can select a 2D image within a 3Dmodel. As depicted, system 200 can comprise a 3D model 208 of a space(e.g., a room). Within the space, a set of 2D images can be available,such as panoramic image 222 and panoramic image 232. Panoramic image 222can be associated with a capture position 224, representing a positionof a camera or other capturing device utilized to capture images thatare stitched together to form panoramic image 222. Likewise, panoramicimage 232 can be associated with a capture position 234, representing aposition of a camera or other capturing device utilized to captureimages that are stitched together to form panoramic image 232.

In an aspect, construction component 110 can generate 3D model 208 inaccordance with various embodiments disclosed herein. In someembodiments, construction component 110 can also generate panoramicimage 222 and panoramic image 232. Selection component 120 can selectpanoramic image 222 or panoramic image 232 based on selection criteriaas described in various embodiments herein. For example, duringnavigation of 3D model 208, selection component can determine a currentposition associated with a current field of view and can selectpanoramic image 222 or panoramic image 232 based on the currentposition. A rendering can switch between rendering the 3D model 208 anda selected panoramic image (e.g., panoramic image 222 or panoramic image232).

In other embodiments, selection component 120 can generate a first setof positions relative to a 3D model and select a 2D image to associatewith the first set of positions. For example, selection component 120can use a metric of distance to determine the 2D image (222, 232)associated with each region of 3D model 208. The minimum distancebetween a given location in the 3D model and the capture position (224,234) of any 2D image (222, 232) may be used as a metric. 2D images canbe associated with positions of a 3D model prior to rendering ornavigation of 3D model 208.

In some embodiments, selection component 120 can utilized weightedscores to select panoramic image 222 or panoramic image 232. Forexample, selection component 120 can select an image based on a weightedscore representing a score determined based on a distance to a captureposition, a quality metric, a user defined preference, or any otherdesired criteria.

Turning now to FIG. 3, a system 300 is depicted. System 300 can capturemedia content and generate correlated models, in accordance with variousembodiments disclosed here. System 300 can include a memory 304 thatstores computer executable components and a processor 302 that executescomputer executable components stored in the memory 304. It is to beappreciated that the system 300 can include various components describedwith reference to other systems or can be used in connection withimplementing one or more of the systems or components shown anddescribed in connection with other figures disclosed herein. It isfurther noted that all or part of system 300 can be included withinlarger systems, such as cameras, tablet computers, smart phones, laptopcomputers, desktop computers, personal digital assistants (PDA's) andthe like.

As depicted, system 300 can include communicably coupled componentsincluding an imagery component 308 (which can generate 3D models basedon captured media and can select 3D data appropriate for given 2Dimagery data). Imagery component 308 can primarily comprise constructioncomponent 310 (which can generate 3D models and 2D models), selectioncomponent 320 (which can select 2D imagery data associated with a givenviewpoint of the 3D model), and capture component 330 (which canfacilitate capturing media content or sensory data). It is noted thatconstruction component 310 and selection component 320 can respectivelyfunction similarly to construction component 110 and selection component120.

Capture component 330 can receive data from one or more image capturingdevices or can comprise one or more image capturing devices. Capturingdevices can comprise 2D camera devices, 3D camera systems, sensors, andthe like. For example, capturing devices can comprise hardware such aslight detection and ranging (LIDAR) sensors, capture devices that notonly capture color but that are also inexpensive, such as thePrimeSense™ Ltd. hardware in Microsoft Corporation's Kinect™, hand-heldlaser line scanners, structured light projectors paired with camerassuch as the Microsoft® Kinect, other structured light systems, stereocameras with software for depth derivation, stereo cameras paired withpattern projection systems as well as software for depth derivation,time-of-flight cameras, video cameras capable of structure-from-motioncalculations, and lightfield cameras. It is further noted that multiplecapture devices may be combined or a capture device may be paired with acolor camera to provide color detail for the captured information.

Turning to FIG. 4, with reference to FIG. 3, there illustrated is anexemplary capturing system 400 in an environment to be captured. System400 can comprise one or more capturing devices, such as device₁ 412 todevice_(N) 414, where N is a number representing a number of cameras.While depicted as separate devices, it is noted that device₁ 412 anddevice_(N) 414 can represent a single devices in one or more positionsover time. In an aspect, device₁ 412 and device_(N) 414 can communicate(e.g., wirelessly, or wired) via a network or can each capture data tobe aggregated at a later time.

In another aspect, device₁ 412 to device_(N) 414 can comprise all or aportion of system 300. As depicted, device₁ 412 can capture images orsensor data from a field of capture 422 and device_(N) 414 can captureimages or sensor data from a field of capture 424. It is noted thatdevice₁ 412 and device_(N) 414 can be rotated about an axis to capturecontent of a real-world space.

In embodiments, device₁ 412 and device_(N) 414 can be rotated orrepositioned via a motor that rotates or repositions device₁ 412 anddevice_(N) 414 such that field of capture 422 and field of capture 424are rotated or repositioned. In some embodiments, the rotation orrepositioning can be accomplished based on a user control, such as inputreceived from input devices (e.g., joystick, buttons, remotes, mouse,etc.) or based on physical movement (e.g., user repositions or manuallyrotates). In an aspect, a set of 2D images or a 2D panoramic image cancorrespond to device₁ 412 and another set of 2D images or a panoramicimage can correspond to device_(N) 414.

In operation, device₁ 412 and device_(N) 414 can capture datacorresponding to one or more views of surface 432 (e.g., a wall),surface 434 (e.g., a different wall), or objects within an environment,such as object 410. It is noted that object 410 can represent anydetectable object, such as furniture, living objects (e.g., plants,people, animals, etc.), or virtually any object. In some embodiments,captured content can be utilized for generation of one or more panoramicimages (e.g., via construction component 310).

Turning back to FIG. 3, with reference to FIG. 4, content captured bycapture component 330 can be received as captured content (e.g., images,videos, sensor data, etc.). In an aspect, capture component 330 canreceived sets of captured content. The sets of captured content cancorrespond to sets based on a particular capturing device (e.g., device₁412 and device_(N) 414), axis point, viewpoint, and the like. Inembodiments, construction component 310 can render models based on thecaptured content, stored content, computer generated content, and thelike. It is noted, that construction component 310 can utilize acombination of different sets of captured content to generate one ormore 3D models or 2D models. In an aspect, construction component 310can correlate the 2D images or 2D viewpoints and the 3D model in acommon coordinate system. In another aspect, selection component 320 canselect 2D images or 2D viewpoints based on a given position in the 3Dmodel.

In embodiments, construction component 310 can generate or render modelsin one or more modes. It is noted that different modes can correspond toseparate models or a single model with varying sets of viewpoints,navigational controls, rendering criteria, and the like. For example, amode can correspond to availability of controls comprising alternatingbetween a set of viewpoints, rotation about one or more axes (e.g.,navigation or motion controls), panning in one or more directions,zooming in or out, or otherwise controlling a field of view. In anotheraspect, rendering criteria can comprise utilizing backface cullingtechniques, floor/ceiling/object removal, and the like. In embodiments,modes can comprise one or more of a 3D mode or a 2D mode, a 3D or 2Dmode of each of a walking mode, a floor plan mode, an orbital mode, orother desired mode. It is noted that references to 2D walking modes,floor plan modes, or orbital modes can comprise 2D representations of 3Dmodes with appropriate modifications or constraints. In someembodiments, a 2D orbital mode may not be available depending onconstraints that define an orbital mode.

A 3D walking mode can refer to a mode that generates viewpoints within a3D model. For example, viewpoints can be based on a camera position, apoint within a 3D model, a camera orientation, and the like. In anaspect, navigation within a 3D model can simulate a user walking throughor otherwise traveling through a real-world scene. The user can rotateand move freely to view the scene from different angles, vantage points,heights, or perspectives. The user may be constrained to have a cameraviewpoint at a particular height above the model surface except whencrouching or in the air (e.g., jumping, falling off an edge etc). In anaspect, collision checking or a navigation mesh can be applied such thatusers are restricted from passing through objects.

In another aspect, a 2D walking mode can refer to a 2D representation ofa 3D walking mode. For example, a 2D walking mode can comprise a set ofpanoramic images and associated viewpoints. A user can navigate betweenviewpoints and rotate about a given viewpoint. It is noted that a 2Dmodel may be constrained so a user can rotate around a single axis.

An orbital mode can represent a mode wherein a user perceives the modelsuch that the user is outside or above the model and can freely rotate amodel about a central point as well as move the central point around themodel. Multiple types of motion may be possible in orbital mode. Forexample, a viewpoint may be pitched up or down, rotated left or rightaround a vertical axis, zoomed in or out, or moved horizontally. Thepitch, rotation-around-a-vertical-axis, and zoom motions may be relativeto a central point, such as defined by an (X, Y, Z) coordinate. Avertical axis of rotation may pass through the central point. In thecase of pitch and rotation-around-a-vertical-axis, those motions maymaintain a constant distance to the central point. Thus, the pitch androtation around-a-vertical-axis motions of the viewpoint may be thoughtof as vertical and horizontal travel, respectively, on the surface of asphere centered on the central point. Zoom may be thought of as travelalong the ray defined as passing through the viewpoint to the centralpoint.

The point on the 3D model with or without back-face culling or otherceiling removal techniques that is rendered in the center of the displaymay be used as the central point. Alternately, this central point may bedefined by the point of a horizontal plane that is at the center of thedisplay. This horizontal plane may be invisible, and its height may bedefined by a global height of the floor of the 3D model. Alternately, alocal floor height may be determined, and the intersection of the raycast from the camera to the center of the display with the surface ofthe local floor height may be used to determine the central point.

The height of the floor of the 3D model may be determined via a varietyof methods. For example, the points or faces of the model with normalvectors that are within a particular threshold angle of verticalpointing up may be assumed to be candidate floor locations. A globalfloor height could be calculated by determining the median of theheights of these candidate floor locations. Alternately, a local floorheight could be computed by taking the average or median of floorheights over a specific area. In an embodiment of orbital mode, theprojection mode for the 3D rendering is perspective mode. Theorientation of the camera may disallow rolling such that the view of themodel is never tilted to one side. Limitations on these and otherpossible modes, orientations, and positions may be imposed. For example,pitch may be clamped such that it is impossible to go past ninetydegrees down at the steepest or some angle above zero degrees down atthe shallowest. The zoom level may be clamped such that the closest zoomis a specified distance from the central point and the farthest zoom issuch that the entire 3D model is visible.

In some embodiments, a 2D orbital mode can comprise a 2D representationof a 3D orbital mode. The 2D orbital mode can comprise navigationconstraints consistent with 2D imagery data. For example, a 2D image ofa floor plan, orthographic view, or perspective view, can be rendered(e.g., by construction component 310). Navigation controls can comprisezooming in/out, moving focus points of a display, altering an angle(pitch) of view, and the like. For example, a 2D image can be thought ofas being a plane in a 3D space. The 2D image can be viewed from above(e.g., at a fixed angle near 90 degrees) or can be viewed from adisparate angle (e.g., less than 90 degrees). In an aspect, a user candetermine an angle of view. A large number of 2D images representingpoints of view of the 3D model may be pre-rendered. Icons representingcaptured 2D imagery may be displayed on top of or near the renderedimage of the 3D model. These icons may allow the user to load and viewcaptured 2D images or panoramas.

A floor plan mode can represent a mode wherein the user perceives themodel such that the user is outside or above the model. For example, auser can view all or a portion of a model from an aerial vantage point.The model can be moved or rotated about an axis. As an example, floorplan mode can correspond to a top down view, wherein the model isrendered such that a user looks directly down onto a model or at a fixedangle down onto the model (e.g., approximately ninety degrees above afloor or bottom plane of a model). The set of motion or navigationcontrols and mappings in floor plan mode may be a subset of those fororbital mode or total available controls of other models. For example,the controls for floor plan mode may be identical to those described inthe context of orbital mode with the exception that the pitch at a fixnumber of degrees downward. Rotation about a central point along avertical axis is still possible as is zooming in and out toward and awayfrom that point and moving the central point. The model may, however,only be viewed directly from above as a result of the fixing a pitch.

In embodiments, 2D modes can include the various modes described above.However, in a 2D mode, rendered content (e.g., by construction component310) can be limited to available 2D imagery data. It is noted that a setof geometric shapes can be generated and 2D imagery can be wrappedaround the geometric shapes such that walls or other objects can bedisplayed in 2D models. It is further noted that a 2D model can comprisea limited set of controls, viewpoints, and the like. For example, a 2Dmodel can comprise a fixed number of viewpoints (e.g., corresponding toa position of capture of 2D images).

Turning now to FIG. 5, a system 500 is depicted. System 500 canfacilitate navigation of 3D models, in accordance with variousembodiments disclosed here. System 500 can include a memory 504 thatstores computer executable components and a processor 502 that executescomputer executable components stored in the memory 504. System 500 canalso include an imagery database 506 (which stores reference media itemsand associated data). While system 500 is described as including imagerydatabase 506, it is noted that system 500 can include other databases,or additional databases.

As depicted, system 500 can include communicably coupled componentsincluding an imagery component 508 (which can generate 3D models basedon captured media and can allow navigation of these models via a mix of3D and 2D modes). Imagery component 508 can primarily compriseconstruction component 510 (which can generate 3D models and 2D models),selection component 520 (which can select 2D imagery data based on 3Dimagery data or navigation data), capture component 530 (which canfacilitate capturing media content or sensory data), and display controlcomponent 540 (which can facilitate navigation of models, transitioningof modes, and the like). It is noted that similarly named components ofsystem 500 and other systems described herein can comprise similaraspects. It is to be appreciated that the system 500 can include variouscomponents described with reference to other systems described herein(e.g., system 100, system 300, etc.).

Imagery database 506 can comprise imagery data (2D or 3D), models,partial models, correlated 2D and 3D imagery, and the like. In anaspect, construction component 510 can receive a model or portion of amodel from imagery database 506 and can render the model. In anotheraspect, capture component 530 can capture content and store the content(as raw data or processed data) in imagery database 506. In embodiments,imagery database 506 can comprise a local storage device or a remotestorage device. For example, a model can be generated or downloaded to alocal storage of a device. In another example, a model or portions of amodel can be streamed to imagery component 508.

In an aspect, display control component 540 can facilitate movementwithin a 3D model, including providing navigation controls, updatingfields of view, generating transitions, generating notifications, andthe like. For example, display control component 540, as described inmore detail with reference to FIG. 6, can provide navigation controlsbased on a 3D model rendered in a particular mode.

A notification can comprise a signal that a particular control orfeature is available or not available. It is noted that a notificationcan comprise an audible cue, visual cue, motion cue (e.g., vibration), acombination of cues, or the like. Audible cues can comprise an audiblesignal output by a device (e.g., speaker), such as a chirp, ring tone,and the like. It is further noted that a set of audible cues can beprovided and a user can selectively determine what each audible cuesignifies (e.g., such as via a user settings interface). Visual cues cantake many forms. Visual cues can comprise activation of a light (e.g.,light emitting diode (LED) of a device), generating a visual cue on aninterface device such as a display screen, and the like. For example, avisual cue can comprise a pop-up or fly-out notification, a token (e.g.,2D symbol, 3D symbol, text data, and the like), modification of an image(e.g., highlighting a portion of a model, flashing a portion of a model,etc.), activation/deactivation of a control (e.g., altering a visualcharacteristic of a button, allowing a button to be selectable, etc.),and the like. A visual cue may be at a fixed point on the display. Insome embodiments, the visual cue may be displayed as an overlay or a 2Dor 3D object in the 3D model. The position of this visual cue relativeto the 3D model may correspond to the correlated position of the part ofthe 3D model seen in the 2D image (e.g., an outline of this region or anicon in the center), the correlated position of the point at which theimage was captured, or the correlated center point from which apanoramic 2D image was captured. This point may also be displayed at thesurface of the model below the center point or capture point. Multiplevisual cues from different 2D images may be displayed at the same time.A particular selection threshold may be used to decide which 2D images'visual cues to display, with only 2D images for which a score meets acertain threshold for selection relative to the current viewpoint of the3D model being displayed.

In embodiments, display control component 540 can be in communicationwith or comprise various input devices. Input devices can, for example,include user interface devices such as a mouse, a keyboard, a touchscreen, a remote, a motion sensor, a microphone, camera, a gesturesensor, or other sensory device. As an example, a user can select acontrol to alternate between viewpoints, zoom in/out of an image,alternate between modes, display 2D content, access menus, and the like.In embodiments, a user can swipe, click, gesture, provide speechcontrols, activate a motion sensor (e.g., shake or rotate a device,etc.), and the like.

In an aspect, selection component 520 can receive a navigation positionfrom display control component 540. The navigation position can comprisedata identifying a position or an orientation corresponding to arendered field of view for a current navigation position within a model,a focal point of a field of view, or the like. For example, withreference to FIG. 2, display control component 540 can facilitatenavigation of 3D model 208. Selection component 520 can receive anavigation position from display control component 540. Based on thenavigation position, selection component 520 can select panoramic image222 or panoramic image 232. It is noted that a selection component 520can select other images or portions of panoramic image 222 or panoramicimage 232. For example, a user can provide input indicating a userdesires to see a single 2D image in a non-panoramic mode in a 2D imageviewing portal (e.g., a pop up window).

Turning now to FIG. 6, a system 600 is depicted. System 600 canfacilitate navigation, manipulation, and control of a model, inaccordance with various embodiments disclosed here. In an aspect, system600 describes a display control component in greater detail. While notshown, system 600 can include a memory that stores computer executablecomponents and a processor that executes computer executable componentsstored in the memory. As depicted, system 600 can include communicablycoupled components including an imagery component 608 (which cangenerate 3D models based on captured media and can allow navigation ofthese models via a combination of 3D and 2D modes). Imagery component608 can comprise a display control component 640 (which can facilitatenavigation of models, transitioning of models, and the like). Displaycontrol component 640 can primarily comprise navigation component 632(which can facilitate navigation of models), notification component 634(which can generate notifications), transitioning component 636 (whichcan transition between modes or 2D/3D imagery), and modificationcomponent 638 (which can provide tools for modifying/altering aspects ofmodels or images). It is to be appreciated that the system 600 caninclude or be in communication with various components described withreference to other systems described herein (e.g., system 100, system300, system 400, system 500 etc.).

Navigation component 632 can facilitate navigation of a model. Inembodiments, navigation component 632 can provide various forms ofnavigation based on a rendered model or available imagery data. Forexample, in a walking mode, navigation component 632 can provide a setof viewpoints representing points where users can move a field of view(e.g., along an axis of an X, Y, Z Euclidian space, along axes relativeto the current orientation, or along the surface of the model at aparticular height), zoom in/out, select an area within a field of view,reposition or reorient a model or imagery data, and the like.

In a walking mode, navigation component 632 can apply a set of availablenavigation controls. Navigation controls can comprise moving along anaxis (e.g., along an axis of an X, Y, Z Euclidian space), moving alongaxes relative to the current orientation, moving along the surface ofthe model at a particular height, altering pitch or orientation, and thelike. In use, a user can utilize controls to alter a field of view, as auser could if the user was walking in a real-world space. This canincluding moving left, right, straight, back, up (e.g., step laddercontrol), down (e.g., crouching), zooming in/out, and the like. It isnoted that backface culling can be applied when generating fields ofview. In another aspect, navigation component 636 can apply limitationsto movements, such as limitations to prevent passing through objects,allowing passing through of certain objects, limitations to movementalong a particular axis, and the like. It is noted, that limitations andavailable controls can be altered based on desired settings, a definedcondition (e.g., subscription to a service), a current mode, and thelike.

Referring now to FIG. 7, with reference to FIG. 6, there illustrated isan example system 700. System 700 can comprise a display 702 that canrender a model and control components. In an aspect, display 702 cancomprise a rendering of a portion of a 3D model comprising object 710,wall 720, wall section 712 for which there is correlated 2D imagery, anda control portal 714. Control portal 714 can comprise one or moreavailable controls such as a navigation control 732, a notificationcontrol 734, a mode control 736, and a modification control 738. It isnoted that display 702 can provide additional controls. Navigationcomponent 632 can navigate within a model such that display 702 alters afield of view. For example, a field of view can rotate such that a usercan see different parts of a living space. In another example, a usercan move closer to object 710.

In another example, navigation component 632 can apply limitations suchthat object 710 cannot be passed through. For example, if a userattempts to “walk” through object 710 then navigation component 632 canprevent the user from passing through object 710 or can trigger anotification that the user is attempting to pass through an image. It isnoted that in some embodiments, navigation component 632 can allow theuser free movement through objects, walls, and the like. In anotheraspect, a particular object or portion of an object can be configured toallow a user to pass through. For example, navigation component 632 candetermine that an object or portion of a model correspond to areal-world door. Identifying the door can comprise image recognition ofthe door, metadata attached to the image, or user input.

In another aspect, navigation component 632 can provide a set ofcontrols for an orbital mode or a floor plan mode. The controls can beappropriately provided based on constraints of the modes. In an aspect,navigation controls can be the same for both orbital mode and floor planmode except for a limitation that the user cannot change a pitch of afield of view in the floor plan mode. Navigation component 632 may alsoprovide for limited navigation of 2D images. For example, rotation,panning, and zooming on a 2D image may be possible. If the 2D image ispanoramic, the user may change their viewpoint's orientation, allowingthem to see different parts of the panorama. While viewing 2D images,users may be able to transition to viewing other 2D images at differentviewpoints or to various viewing modes for the 3D model. Users may havea set of navigation controls 732 allowing these actions to take place.

In embodiments, navigation component 632 can facilitate manualnavigation (e.g., based on user input), automatic navigation, orsemi-automatic navigation. In manual navigation, a field of view doesnot change unless a user provides input. In automatic navigation, afield of view can change based on a predefined pattern, a dynamicallydefined pattern, and the like. In semi-automatic navigation, a field ofview can change automatically or according to user input. For example,in a semi-automatic mode, navigation can be automatic unless a userprovides input to interrupt, manipulate, or alter an automaticnavigation. In another example, navigating to particular portions of amodel or navigating to the portion for an amount of time can trigger anautomatic navigation of the portion. For example, if a user navigates toa room of a 3D model, an automatic navigation of the room can betriggered. In another example, if the user does not provide input for adetermined amount of time, then the navigation can switch to anautomatic mode.

In another aspect, navigation component 632 can generate a video clip orslide show of a navigation of a model. For example, different viewpointscan be captured during navigation and the captured viewpoints can beaggregated into a video clip, slideshow, or other media content. In someembodiments, a user can provide input indicating the user desires thatthe navigation component 632 automatically generate media content basedon navigation (manual, automatic, or semi-automatic). The media contentcan be generated based on an entire model or a portion of a model. Forexample, during manual navigation a user can provide input indicatingthe user would like a video of navigation of a particular room of a 3Dmodel. Navigation component 632 can generate a video of manual usernavigation or generate a video of an automatic navigation of the room.

Notification component 634 can generate notifications via one or moreinterfaces. As noted supra, a notification can comprise an audio,visual, textual, or motion notification. In embodiments, notificationcomponent 634 can generate notifications based on triggering events. Atriggering event can comprise user input, navigation to a field of viewor portion of a model, mousing over or hovering over an object,selecting an object (e.g., selecting a button, selecting an image,selecting a portion of a model, etc.), determining that a 2D image isavailable for view, and the like. As an example, a user can navigate tofield of view 702. Notification component 634 can generate anotification indicating that 2D imagery data associated with wallsection 712 can be viewed in the field of view. In an aspect, wallsection 712 can correspond to a 2D image with a known position andorientation within the 3D model. The user can then select to view the 2Dimage and system 600 can render the 2D image (e.g., via transitioningcomponent 636), as described below. In an aspect, notification component634 can generate the notification in notification control 734. Inanother aspect, the notification can be outlining image 712,highlighting image 712, a generated indicator such as a notificationtoken or faded 2D image (e.g., a graphical notification), a pop-upwindow, and the like. It is noted that a notification token can beplaced at the floor (e.g., on a planar barrier) below a portion of amodel that corresponds to an available 2D image, hovering a determinedheight above the floor, and the like.

In another example, an audible tone can be generated or a motioncomponent can generate a vibration signal. The audible tone or vibrationcan indicate that a 2D image is available for view within the field ofview.

In another aspect, notification component 634 can generate anotification based on a user viewing a field of view for a determinedamount of time. For example, a user can view field of view 702 for ameasured amount of time. If the measured amount of time exceeds athreshold, a notification can be generated.

In an orbital mode or floor plan mode, notification component 634 cangenerate one or more notifications associated with any available 2Dimagery data available for view. As an example, fly outs or symbols canbe rendered within a model; these symbols may correspond to locationswith available 2D imagery data or may be situated at the capture pointof this 2D imagery data relative to the 3D model.

It is noted that notification component 634 can generate notificationsfor various events, such as attempting to pass through an object,switching modes, available viewpoints of a 2D image, availability ofpanoramic images, availability of video, availability of audio, and thelike. As such, examples described herein are not intended to beinclusive of every possible triggering event or notification. It isfurther noted that virtually any event can be configured fornotification component 634 to trigger generation of a notification.

In further aspects, notification component 634 can generate a list orsummary of available 2D or 3D images that can be viewed (e.g., such asselected 2D images by selection component 120, selection component 320,or selection component 520, all nearby 2D images, or all available 2Dimages). In an aspect, the list can correspond to triggering events. Itis noted that the list can be displayed, for example, in notificationcontrol 734. Displaying the list can comprise displaying a partial listbased on a field of view, a room, a defined area, user input, displayingtext data, displaying a set of token markers on a model (e.g., inorbital mode or a floor plan mode all or a set of markers can bedisplayed), and the like. In some embodiments, the list can be expanded,displayed in a disparate screen or window, and the like.

Transitioning component 636 can facilitate transitions between modes. Amode can be transitioned from, for example, a walking mode to a floorplan mode. It is noted that during transitioning between modes, backface culling, ceiling removal, wall removal, or other techniques can beturned on or off or otherwise changed during transitions. A smoothtransition can comprise panning from a first viewpoint to a secondviewpoint. In another aspect, transitioning can comprise instanttransitions, external transitions (e.g., in separate windows), partialtransitions, and the like. Instant transitions can be, for example,“snap” transitions or “jump to” transitions where a model or field ofview is switched instantaneously. It is noted that instantaneously doesnot necessarily mean without any delay. For example, a processing delay(e.g., depending on a processing speed) or a notification delay. It isnoted that transitioning component 636 can transition between any modes.

In embodiments, transitioning can be based on a triggering event. Thetriggering event can comprise virtually any desired event, such as userinput (e.g., selecting of a mode control 736, selection of notificationcontrol 734, selection of an object (e.g., object 710), selection of animage (e.g., image 712, etc.), passage of time, navigating to aparticular point (or set of points) of a model, selection of an iconcorresponding to available 2D imagery, and the like. Selecting an imageor object can comprise viewing the object for a determined amount oftime, zooming in on an object/image, swiping, tapping, or otherwiseindicating a desire to select an image/object. In embodiments,transitioning component 636 can facilitate transitioning between 2Dmodels or images and 3D models. Transitioning between 2D models orimages and 3D models can comprise a complete transition between 3D and2D models or images, transitioning between portions of 2D models orimages overlaying 3D models, or the like. As an example, a user can benavigating or viewing a 3D model (e.g., in walking mode, orbital mode,floor plan mode, etc.) and in response to occurrence of a triggeringevent, transitioning component 636 can facilitate display of appropriate2D imagery. It is noted that transitioning can be automatic or manual.For example, in a manual transitioning mode a triggering event cancomprise receiving user input. In an automatic transitioning mode, atriggering event can comprise navigation to a determined area of the 3Dmodel, dwelling near that point for a specific period of time, and thelike.

Transitioning component 636 can select imagery data to be transitionedto based on an appropriate selection of 2D imagery data (e.g., such asselected 2D images by selection component 120, selection component 320,or selection component 520). For example, a 2D image (e.g., image 712)can be selected as the best match to a viewpoint of a 3D model. Inresponse to occurrence of a triggering event, transitioning component636 can generate a 2D image. Generating the 2D image can comprisedisplaying a 2D panoramic image to replace a 3D display, displaying a 2Dimage over at least a portion of 3D model, generating a pop up view of a2D image, and the like.

In embodiments, transitioning component 636 can transition betweenviewpoints associated with 2D images. For example, a user can select toview a 2D image within a 3D model. Transitioning component 636 cantransition the user's view to a selected 2D image. In an aspect,selection of the 2D viewpoint can be based on a similar field of view,minimizing a distance to a 2D viewpoint, selection of a tokennotification indicating a position of a 2D viewpoint, and the like. Itis noted that a distance can be an absolute distance or a navigationaldistance (e.g., distance around objects, walls, or barriers).

In some embodiments, transitioning component 636 can generate visualrays that indicate a location of a 2D image in a 3D model. For example,a 2D image can be represented as an overlay with a 3D model displayedbelow. A line or shading can be drawn from the 2D image to a point onthe 3D model. In another aspect, a 3D model can be dimmed or otherwisealtered to draw attention to the 2D image.

In another aspect, transitioning component 636 can transition to andfrom different viewpoints. For example, a user can transition from a 3Dmodel to view a 2D image at a correlated viewpoint. This transition mayinvolve moving the viewpoint from the pre-transition viewpoint of the 3Dmodel to the position of the viewpoint from which the 2D image wascaptured as correlated to the 3D model. Once this movement is complete,the view of the 3D model may switch or fade to the corresponding view ofthe 2D image. When the user desires to switch back to the 3D model,transitioning component 636 can transition the user back to a previous3D viewpoint prior to the transition. In other embodiments,transitioning component 636 can transition to a viewpoint that is common(e.g., a distance from the correlated viewpoint of the 2D image in the3D model to the viewpoint provided after the transition back to 3D iszero or about zero) between different modes.

Transitioning component 636 can transition between various modes.Transitioning between modes can comprise smooth transitions, snaptransitions, and the like. In an aspect, during a smooth transition,parameters of a viewpoint can be smoothly transitioned, such as smoothlymoving, avoiding barriers, passing through barriers, smoothly rotating,and the like. It is noted that a speed of transition can be configured(e.g., the rate of rotation or movement) according to a desired set ofparameters. Transitioning component 636 can initiate a transition, forexample, based on a triggering event. A triggering event can correspondto user input (e.g., selection of mode control 736), panning in/out to athreshold, and the like. In an aspect, transition component 636 canapply a transition delay to avoid accidental switching between modes,generate a notification that may ask for user confirmation, and thelike.

In embodiments, transitioning component 636 can determine a targettransition point for a transition. For example, when a transition istriggered between a first mode and a second mode, the transitioningcomponent 636 can determine a focal point for a display of the secondmode as a central point of a target transition viewpoint. In an aspect,the target transition viewpoint can be determined based on a currentposition or associated focal point. For example, a central point of adisplay can be identified as a transition point and the transition pointcan be correlated for the second viewpoint such that the central pointremains in a center of the display. In another example, a user canselect a desired transition point (e.g., select a point on a floor planmode). It is noted that a transition points can be fixed transitionpoints or predefined.

Modification component 638 can facilitate manual or automaticmodification of imagery data or a rendering of a model. In embodiments,modification component 638 can provide a set of controls (e.g.,modification control 738) to a user that can facilitate modifications toa model or images. The set of controls can comprise, for example,building tools (which can add/remove or alter objects, images, barriers,etc.), appearance modification tools (which can change the color ortexture of a surface or object), image enhancing tools (e.g. noisereduction tools), user upload tools, settings configuration tools,annotation tools, virtual tour tools, measurement tools, and the like.

A measurement tool can facilitate measuring of objects, rooms, barriers,volumetric space, area, and the like. In an aspect, a user can selectstart and endpoints for measurements (e.g., through an input device) onthe 3D model and modification component 638 can determine a measurementbased on the start and end points. In another aspect, a user can selectan area or object based on a lasso tool or based on drawing a border,for example.

In some embodiments, a user can select an object and modificationcomponent 638 can automatically determine measurement data associatedwith the object. For example, a user can select a piece of furniture formeasurement. Modification component 638 can identify and measure thepiece of furniture based on image recognition (e.g., identifying commonfeatures), analysis of a mesh, analysis of point data, and the like. Itis noted that modification component 638 can draw measurements on adisplay, provide a list of measurements, or otherwise presentmeasurements. It is further noted that measurements can be stored orcorrelated between models, between 3D and 2D data, and the like.

An annotation tool of modification component 638 can facilitategenerating notes or annotations on 2D or 3D imagery data. An annotationcan comprise textual or other data that is connected to a particularobject, barrier, or point of a model or image. It is noted that if 2Dand 3D imagery data is correlated, the annotations can be attached toboth 2D and 3D imagery data. It is further noted that modificationcomponent 638 can save annotations, draw annotations on a display,generate a list of annotations, or otherwise present annotations.

Building tools of modification component 638 can facilitate adding,removing, altering, orienting, or otherwise manipulating aspects of amodel or image. In an aspect, modification component 638 can supply alist of pre-defined 2D images or 3D objects that can be added to a modelor image. For example, a user can add a 3D piece of furniture to a 3Dmodel. The furniture can be positioned based on user. In an aspect,positioning can be constrained to avoid collisions and the like. Inanother aspect, modification component 638 receive user generated(uploaded) 2D images or 3D objects that can be added to a model orimage. In another aspect, modification component 638 can provide a setof configurable objects. A user can select characteristics of theobjects (e.g., color, length, depth, height, etc.) and can add theobject or save an object for later use.

In embodiments, system 600 can receive input representing imagescaptured by a user (e.g., via a camera, or the like). For example, auser can upload a 2D image, which can be correlated with a 3D model todetermine its viewpoint. The data from this 2D image may also be used toupdate the texture of the 3D model.

Building tools of modification component 638 can further comprise imageor object manipulation capability. For example, a user can select asurface and can alter the color of a surface. In another example, a usercan select a portion of an object or image to be removed or cropped. Inanother aspect, modification component 638 can allow for alteringresolution or image channels of 2D or 3D imagery.

While the modifications described herein take place in the space of the3D model and are rendered as part of the process or as an addendum torendering the 3D model, they may be also rendered as an overlay on 2Dimagery when in a 2D display mode. In an embodiment, this may befacilitated by an associated depth map that can be determined for a 2Dimage. The depth map may contain depth data that is known or generatedper pixel. In an aspect, the depth map can be generated by capturecomponent 330 or it can be derived by rendering the 3D model with aviewpoint at the correlated capture point of the 2D image and noting thedistance to each pixel of the rendered model. System 600 can utilize thedepth map to appropriately overlay or occlude modifications to the 3Dmodel on the 2D image. For example, measurements and annotations can bedisplayed at the correct points. Supplemental 3D objects (such asuser-added 3D objects or markers denoting panorama centers) could bedisplayed in the 2D image, with the correct parts of the 2D imageoccluding the 3D objects if they are nearer to the viewpoint than theobject at that point in the image. Portions of the 3D model withmodified colors, textures, or surface appearances may be similarlyoverlaid on the 2D image based on this depth masking of the 2D image.The distance values in the depth map may be increased slightly to dealwith the situation in which modified parts of the 3D model and the depthmap are at almost exactly the same depth; this increase will allow anymodifications to the 3D model to be drawn over corresponding parts ofthe 2D image.

FIGS. 8-12 illustrate various methodologies in accordance with certainembodiments of this disclosure. While, for purposes of simplicity ofexplanation, the methodologies are shown media a series of acts withinthe context of various flowcharts, it is to be understood andappreciated that embodiments of the disclosure are not limited by theorder of acts, as some acts may occur in different orders orconcurrently with other acts from that shown and described herein. Forexample, those skilled in the art will understand and appreciate that amethodology can alternatively be represented as a series of interrelatedstates or events, such as in a state diagram. Moreover, not allillustrated acts may be required to implement a methodology inaccordance with the disclosed subject matter. Additionally, it is to befurther appreciated that the methodologies disclosed hereinafter andthroughout this disclosure are capable of being stored on an article ofmanufacture to facilitate transporting and transferring suchmethodologies to computers. The term article of manufacture, as usedherein, is intended to encompass a computer program accessible from anycomputer-readable device or storage media. It is noted that the methodsdepicted in FIGS. 8-12 can be performed by various systems disclosedherein, such as systems 100, 200, 300, 400, 500, and 600.

FIG. 8 illustrates exemplary method 800. Method 800 can provide forgenerating and transitioning between 3D imagery data and 2D imagerydata. In another aspect, the method 800 can select 2D imagery thatcorresponds to a viewpoint associated with 3D imagery and render a modelwith correlated 2D and 3D imagery. For example, a system can generate a3D model and can correlate 2D data with the 3D model. The system cantransition between displaying 3D imagery and 2D imagery.

At reference numeral 802, a system can generate (e.g., via constructioncomponent 110) one or more models based on imagery data. Imagery datacan be received from capturing devices or from a memory store.Generating a model can comprise generating a 2D model, 3D model, orvarious types of 2D or 3D models. For example, a system can generate a3D model comprising a 3D mesh, point data, or color data. In anotheraspect, a system can generate a 2D model based on stitching 2D imagesand the like.

At 804, the system can select (e.g., via selection component 120) 2Dimagery data based on 3D imagery data and associated viewpoints or basedon navigational positions. Selecting 2D imagery can comprise determiningcontent matches (e.g., based on image identification, bitwise contentcomparison, etc.) between 2D imagery data and 3D imagery data,determining matched position data (e.g., a capture point of a 2D imageand a viewpoint associated with a 3D imagery data), and the like. Inanother aspect, selecting 2D imagery data can comprise positioning 2Ddata within a 3D model. As an example, a system can identify 2D imagesand select, based on selection criteria, one or more 2D images thatmatch a viewpoint or portion of a 3D model. Matched 2D images canreceive positional data corresponding to a 3D model, such that thematched 2D images can be displayed in a 2D mode based on a positionrelative to a 3D model. It is noted that in some embodiments matched 2Dimages can comprise data correlating the 2D imagery data and 3D imagerydata (e.g., as a result of constructing a 3D model). It is noted thatthe selecting can take place prior to, during, or after rendering amodel.

At 806, a system can render (e.g., via construction component 110) amodel. A model can be rendered in one or more modes. For example, a 3Dmodel can be rendered in a walking mode, an orbital mode, a floor planmode, and the like. In an aspect, 2D models can be rendered as a set ofpanoramic images, a set of flat 2D images, a slide show, and the like.In an aspect, rendering can comprise generating instructions tofacilitate an output device displaying content. In another aspect,rendering can comprise updating a model as a user navigates a model andthe like. As described supra, a system can render a model at 806 priorto selecting 2D imagery data at 804. For example, a system can render a3D model can select 2D images for display during rendering or navigationof a model. In another aspect, a system can preselect a first set of 2Dimages before rendering or navigation of a model and can dynamicallyselect a second set of 2D images after or during rendering or navigationof the model.

At 808, a system can, in response to a trigger, transition (e.g., viatransitioning component 636) between 3D imagery data and 2D imagerydata. In embodiments, a system can detect a trigger based on occurrenceof a triggering event. For example, the triggering event can comprisedata representing user input (e.g., from a touch screen device, a mouse,a keyboard, a button, a microphone, a camera, or other interfacedevice), navigation to a particular point of a model (e.g., a viewpointis a threshold level away from an object), passage of time (e.g.,viewing a viewpoint for a determined amount of time), based on a historyof use (e.g., learning user behavior), a quality metric, and the like.For example, a system can determine if the 3D model comprises holesrepresenting incomplete data sets or defects in quality of 3D imagerydata. A system can determine the presence of a hole or defect in a 3Dmodel and in response can trigger transitioning to select 2D imagerydata associated with a position, within a 3D model, of the hole ordefect.

In some embodiments, 2D images can be selected in response to a trigger.For example, a 2D image can be selected based on a user navigating to aposition of a 3D model, user input indicating a desire to view 2D imagesand the like. As described in various embodiments herein, selecting a 2Dimage(s) can comprise determining a best image from a set of candidateimages based on a distance to viewpoints of images, image quality, andthe like.

In embodiments, transitioning between 3D imagery data and 2D imagerydata can comprise rendering 3D imagery data then transitioning torendering 2D imagery data. It is noted that transitioning to rendering2D imagery data can comprise rendering only 2D imagery data, rendering2D and 3D imagery data, and the like. For example, a system can render a3D model and transition to a 2D panoramic image. In another example, asystem and render a 3D model and transition to a 2D view portal within a3D model. A 2D view portal can comprise a 2D image over a 3D image (2Doverlay), a 2D slide show or composite image over a 3D image, a 2D imagein a pop-out window, navigation window, or disparate screen, and thelike. In an aspect, rendering a 2D overlay can comprise altering a 3Dmodel that is displayed below a 2D overlay. For example, the 3D modelcan be dimmed, grayed, and the like. In another aspect, controls of a 3Dmodel can be locked or disabled, such as disabling navigation controls.

Turning now to FIG. 9, exemplary method 900 is depicted. Method 900 canprovide for selecting 2D imagery data based on navigational informationand a field of view. For example, a system can facilitate a usernavigating a 3D model. The system can select and display a 2D imagebased on the user's current position within the model and a field ofview as rendered by a display.

At reference numeral 902, a system can facilitate navigation (e.g., vianavigation component 632) of a 3D model. In an aspect, navigation caninclude providing one or more controls to enable a user to move about a3D model in accordance with various embodiments disclosed herein.

At 904, a system can determine (e.g., via navigation component 632) dataregarding a current position within a model and a field of view. It isnoted that a system can utilize various techniques to determine a user'sposition within the 3D model. In another aspect, determining a field ofview can include determining an area of a 3D model, which is currentlyrendered, determining a focal point of a rendering, and the like. Inanother aspect, determining the current position and a field of view canbe based on data received from a capturing devices.

At 906, a system can select (e.g., via selection component 320) a 2Dimage based on the current position and the field of view. Inembodiments, a system can utilize one or more selection factors todetermine a best image from a set of images associated with position andthe field of view.

At 908, a system can transition (e.g., via transition component 636)between rendering the 3D model and rendering the 2D image. Transitioningbetween rendering the 3D model and rendering the 2D image can compriserendering the 2D image in place of the 3D model, rendering a combinationof a 3D model and 2D images, and the like.

FIG. 10 illustrates exemplary method 1000. Method 1000 can provide fortransitioning from a 3D model to a 2D model in a 3D modeling system. Inanother aspect, the method 1000 can determine when to transition betweenmodels. Method 1000 can initiate at reference numeral 1002, by rendering(e.g., by a system via display control component 640) a 3D model.

At 1004, a system can generate (e.g., via notification component 634) anotification indicating availability of 2D imagery data. In embodiments,the system generates the notification in response to occurrence of anevent. The event can be user input indicating a desire to see available2D images, navigation to a position in a 3D model, viewing a field offield, passage of time, and the like. As an example, a user can navigatea 3D model. If the user approaches a particular object and is within adetermined threshold distance, then a notification can be generated. Insome aspects, a user must remain within a field of view (e.g., field ofview can vary within a tolerance level) for a determined amount of timebefore a notification is generated. This can prevent unwantednotifications when navigating a model.

At 1006, a system can detect (e.g., via transitioning component 636 ornotification component 634) a trigger indicating a desire to transitionbetween 3D and 2D imagery data. As mentioned above, the trigger can beany number of triggers, such as user input and the like.

At 1008, a system can select (e.g., via transitioning component 636)appropriate 2D imagery data. In an aspect, 2D imagery can be selectedbased on a predetermined set of candidate images, navigationaldistances, quality metrics, and the like. In another aspect, a systemcan select appropriate 2D imagery data based on content matching orbased on raw 2D imagery data associated with 3D imagery data. At 1010, asystem can transition (e.g., via transitioning component 636) toselected 2D imagery viewpoint and render 2D imagery.

Turning now to FIG. 11, exemplary method 1100 is depicted. Method 1100can provide for transitioning between a first mode and a second mode.For example, the method 1100 can provide for rendering a model in a 3Dwalking mode to rendering a model in an orbital mode, a floor plan mode,or a 2D mode.

At 1102, the system can render (e.g., via construction component 510) amodel in a first mode. For example, a system can render a model in a 3Dmode, a 2D mode, or respective 3D and 2D walking modes, orbital modes,or floor plan modes.

At 1104, a system can receive (e.g., via display control component 540)an instruction to alternate between the first mode to a second mode. Itis noted that the instruction can be based on occurrence of an event,such as a triggering event. At 1106, a system can, in response to theinstruction, transition (e.g., via display control component 540) to arendering of the second mode. It is noted that the second model cancomprise a 3D mode, a 2D mode, or respective 3D and 2D walking modes,orbital modes, or floor plan modes.

In embodiments, transitioning between modes can comprise determining amethod of transitioning, such as a smooth or snap transition. It isnoted that the method can be predetermined or based on user input orsettings. In an aspect, smooth transitioning can comprise determining adestination viewpoint and a path to a destination viewpoint. In anotheraspect, smooth transitioning can comprise smoothly moving or rotating toa destination viewpoint from an origin viewpoint.

In some embodiments, transitioning between a 3D model and a 2D model cancomprise selecting appropriate images and viewpoints to render.Rendering can comprise generating pop-outs, overlays, and the like. Inanother aspect, transitioning can comprise rendering an image in a sideview menu while maintaining the rendering of the first mode.

Turning now to FIG. 12, exemplary method 1200 is depicted. Method 1200can provide for enhancing a view of a 3D model based on user input. Forexample, the method 1200 can facilitate enhancing a view of a portion ofa 3D model to a maximum or near maximum available fidelity in a 2Dimage.

At 1202, a system can receive (e.g., via display control component 540)input representing a user's selection of a region or object in arendering of a 3D model. For example, a system can generate a renderingof a view of a portion of the 3D model and a user can provide input(e.g., via an interface) comprising information indicating the userdesires a region or object in the rendering to be enhanced.

At 1204, a system can select (e.g., via selection component 520) 2Dimagery data based on the received input. For example, the system canselect 2D imagery data corresponding to a user selected region orobject. In an aspect, the 2D imagery data can be selected based on aselection criterion or selection criteria. The selection criterion canfacilitate selection of 2D imagery data having the most detailed view ofa selected area or object. It is noted that the selection can comprise ascoring process that generates scores based on criteria such asmaximizing visual fidelity in absolute spatial terms and maximizing thevisible portion of the region or object as viewed in the 3D model thatis also present in the 2D imagery data. In some embodiments, the systemcan crop the 2D imagery data to only select the desired region ofinterest.

At 1206, a system can transition (e.g., via transitioning component 636)between rendering a 3D view and rendering a 2D view. The transition cantake the form of a user-selected region or object expanding in 2D tofill a portion of a display in accordance with various embodimentsdisclosed herein. In an aspect, the expansion can be smooth and thetransition from the image of the section of the 3D model to the 2Dimagery data may be gradual (e.g., a fade). While expanding auser-selected region or object is described above, it is noted that thetransition can take one or more forms as described in variousembodiments herein.

The systems and processes described below can be embodied withinhardware, such as a single integrated circuit (IC) chip, multiple ICs,an application specific integrated circuit (ASIC), or the like. Further,the order in which some or all of the process blocks appear in eachprocess should not be deemed limiting. Rather, it should be understoodthat some of the process blocks can be executed in a variety of orders,not all of which may be explicitly illustrated herein.

With reference to FIG. 13, a suitable environment 1300 for implementingvarious aspects of the claimed subject matter includes a computer 1302.The computer 1302 includes a processing unit 1304, a system memory 1306,a codec 1335, and a system bus 1308. The system bus 1308 couples systemcomponents including, but not limited to, the system memory 1306 to theprocessing unit 1304. The processing unit 1304 can be any of variousavailable processors. Dual microprocessors and other multiprocessorarchitectures also can be employed as the processing unit 1304.

The system bus 1308 can be any of several types of bus structure(s)including the memory bus or memory controller, a peripheral bus orexternal bus, or a local bus using any variety of available busarchitectures including, but not limited to, Industrial StandardArchitecture (ISA), Micro-Channel Architecture (MSA), Extended ISA(EISA), Intelligent Drive Electronics (IDE), VESA Local Bus (VLB),Peripheral Component Interconnect (PCI), Card Bus, Universal Serial Bus(USB), Advanced Graphics Port (AGP), Personal Computer Memory CardInternational Association bus (PCMCIA), Firewire (IEEE 1394), and SmallComputer Systems Interface (SCSI).

The system memory 1306 includes volatile memory 1310 and non-volatilememory 1312. The basic input/output system (BIOS), containing the basicroutines to transfer information between elements within the computer1302, such as during start-up, is stored in non-volatile memory 1312. Inaddition, according to present innovations, codec 1335 may include atleast one of an encoder or decoder, wherein the at least one of anencoder or decoder may consist of hardware, software, or a combinationof hardware and software. Although, codec 1335 is depicted as a separatecomponent, codec 1335 may be contained within non-volatile memory 1312.By way of illustration, and not limitation, non-volatile memory 1312 caninclude read only memory (ROM), programmable ROM (PROM), electricallyprogrammable ROM (EPROM), electrically erasable programmable ROM(EEPROM), or flash memory. Volatile memory 1310 includes random accessmemory (RAM), which acts as external cache memory. According to presentaspects, the volatile memory may store the write operation retry logic(not shown in FIG. 13) and the like. By way of illustration and notlimitation, RAM is available in many forms such as static RAM (SRAM),dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM(DDR SDRAM), and enhanced SDRAM (ESDRAM).

Computer 1302 may also include removable/non-removable,volatile/non-volatile computer storage medium. FIG. 13 illustrates, forexample, disk storage 1314. Disk storage 1314 includes, but is notlimited to, devices like a magnetic disk drive, solid state disk (SSD)floppy disk drive, tape drive, Jaz drive, Zip drive, LS-100 drive, flashmemory card, or memory stick. In addition, disk storage 1314 can includea storage medium separately or in combination with other storage mediaincluding, but not limited to, an optical disk drive such as a compactdisk ROM device (CD-ROM), CD recordable drive (CD-R Drive), CDrewritable drive (CD-RW Drive) or a digital versatile disk ROM drive(DVD-ROM). To facilitate connection of the disk storage devices 1314 tothe system bus 1308, a removable or non-removable interface is typicallyused, such as interface 1316. It is appreciated that storage devices1314 can store information related to a user. Such information might bestored at or provided to a server or to an application running on a userdevice. In one embodiment, the user can be notified (e.g., by way ofoutput device(s) 1336) of the types of information that are stored todisk storage 1314 or transmitted to the server or application. The usercan be provided the opportunity to control use of such informationcollected or shared with the server or application (e.g., by way ofinput from input device(s) 1328).

It is to be appreciated that FIG. 13 describes software that acts as anintermediary between users and the basic computer resources described inthe suitable operating environment 1300. Such software includes anoperating system 1318. Operating system 1318, which can be stored ondisk storage 1314, acts to control and allocate resources of thecomputer system 1302. Applications 1320 take advantage of the managementof resources by operating system 1318 through program modules 1324, andprogram data 1326, such as the boot/shutdown transaction table and thelike, stored either in system memory 1306 or on disk storage 1314. It isto be appreciated that the claimed subject matter can be implementedwith various operating systems or combinations of operating systems.

A user enters commands or information into the computer 1302 throughinput device(s) 1328. Input devices 1328 include, but are not limitedto, a pointing device such as a mouse, trackball, stylus, touch pad,keyboard, microphone, joystick, game pad, satellite dish, scanner, TVtuner card, digital camera, digital video camera, web camera, and thelike. These and other input devices connect to the processing unit 1304through the system bus 1308 via interface port(s) 1330. Interfaceport(s) 1330 include, for example, a serial port, a parallel port, agame port, and a universal serial bus (USB). Output device(s) 1336 usesome of the same type of ports as input device(s) 1328. Thus, forexample, a USB port may be used to provide input to computer 1302 and tooutput information from computer 1302 to an output device 1336. Outputadapter 1334 is provided to illustrate that there are some outputdevices 1336 like monitors, speakers, and printers, among other outputdevices 1336, which require special adapters. The output adapters 1334include, by way of illustration and not limitation, video and soundcards that provide a means of connection between the output device 1336and the system bus 1308. It should be noted that other devices orsystems of devices provide both input and output capabilities such asremote computer(s) 1338.

Computer 1302 can operate in a networked environment using logicalconnections to one or more remote computers, such as remote computer(s)1338. The remote computer(s) 1338 can be a personal computer, a server,a router, a network PC, a workstation, a microprocessor based appliance,a peer device, a smart phone, a tablet, or other network node, andtypically includes many of the elements described relative to computer1302. For purposes of brevity, only a memory storage device 1340 isillustrated with remote computer(s) 1338. Remote computer(s) 1338 islogically connected to computer 1302 through a network interface 1342and then connected via communication connection(s) 1344. Networkinterface 1342 encompasses wire or wireless communication networks suchas local-area networks (LAN) and wide-area networks (WAN) and cellularnetworks. LAN technologies include Fiber Distributed Data Interface(FDDI), Copper Distributed Data Interface (CDDI), Ethernet, Token Ringand the like. WAN technologies include, but are not limited to,point-to-point links, circuit switching networks like IntegratedServices Digital Networks (ISDN) and variations thereon, packetswitching networks, and Digital Subscriber Lines (DSL).

Communication connection(s) 1344 refers to the hardware/softwareemployed to connect the network interface 1342 to the bus 1308. Whilecommunication connection 1344 is shown for illustrative clarity insidecomputer 1302, it can also be external to computer 1302. Thehardware/software necessary for connection to the network interface 1342includes, for exemplary purposes only, internal and externaltechnologies such as, modems including regular telephone grade modems,cable modems and DSL modems, ISDN adapters, and wired and wirelessEthernet cards, hubs, and routers.

Referring now to FIG. 14, there is illustrated a schematic block diagramof a computing environment 1400 in accordance with this specification.The system 1400 includes one or more client(s) 1402 (e.g., laptops,smart phones, PDAs, media players, computers, portable electronicdevices, tablets, and the like). The client(s) 1402 can be hardware,software, or hardware in combination with software (e.g., threads,processes, computing devices). The system 1400 also includes one or moreserver(s) 1404. The server(s) 1404 can also be hardware or hardware incombination with software (e.g., threads, processes, computing devices).The servers 1404 can house threads to perform transformations byemploying aspects of this disclosure, for example. One possiblecommunication between a client 1402 and a server 1404 can be in the formof a data packet transmitted between two or more computer processeswherein the data packet may include video data, resources referencingmedia items, and the like. The data packet can include a cookie orassociated contextual information, for example. The system 1400 includesa communication framework 1406 (e.g., a global communication networksuch as the Internet, or mobile network(s)) that can be employed tofacilitate communications between the client(s) 1402 and the server(s)1404.

Communications can be facilitated via a wired (including optical fiber)or wireless technology. The client(s) 1402 are operatively connected toone or more client data store(s) 1408 that can be employed to storeinformation local to the client(s) 1402 (e.g., cookie(s) or associatedcontextual information). Similarly, the server(s) 1404 are operativelyconnected to one or more server data store(s) 1410 that can be employedto store information local to the servers 1404.

In one embodiment, a client 1402 can transfer an encoded file, inaccordance with the disclosed subject matter, to server 1404. Server1404 can store the file, decode the file, or transmit the file toanother client 1402. It is to be appreciated, that a client 1402 canalso transfer uncompressed file to a server 1404 and server 1404 cancompress the file in accordance with the disclosed subject matter.Likewise, server 1404 can encode video information and transmit theinformation via communication framework 1406 to one or more clients1402.

The illustrated aspects of the disclosure may also be practiced indistributed computing environments where certain tasks are performed byremote processing devices that are linked through a communicationsnetwork. In a distributed computing environment, program modules can belocated in both local and remote memory storage devices.

Moreover, it is to be appreciated that various components describedherein can include electrical circuit(s) that can include components andcircuitry elements of suitable value in order to implement theembodiments of the subject innovation(s). Furthermore, it can beappreciated that many of the various components can be implemented onone or more integrated circuit (IC) chips. For example, in oneembodiment, a set of components can be implemented in a single IC chip.In other embodiments, one or more of respective components arefabricated or implemented on separate IC chips.

What has been described above includes examples of the embodiments ofthe present invention. It is, of course, not possible to describe everyconceivable combination of components or methodologies for purposes ofdescribing the claimed subject matter, but it is to be appreciated thatmany further combinations and permutations of the subject innovation arepossible. Accordingly, the claimed subject matter is intended to embraceall such alterations, modifications, and variations that fall within thespirit and scope of the appended claims. Moreover, the above descriptionof illustrated embodiments of the subject disclosure, including what isdescribed in the Abstract, is not intended to be exhaustive or to limitthe disclosed embodiments to the precise forms disclosed. While specificembodiments and examples are described herein for illustrative purposes,various modifications are possible that are considered within the scopeof such embodiments and examples, as those skilled in the relevant artcan recognize. Moreover, use of the term “an embodiment” or “oneembodiment” throughout is not intended to mean the same embodimentunless specifically described as such.

In particular and in regard to the various functions performed by theabove described components, devices, circuits, systems and the like, theterms used to describe such components are intended to correspond,unless otherwise indicated, to any component which performs thespecified function of the described component (e.g., a functionalequivalent), even though not structurally equivalent to the disclosedstructure, which performs the function in the herein illustratedexemplary aspects of the claimed subject matter. In this regard, it willalso be recognized that the innovation includes a system as well as acomputer-readable storage medium having computer-executable instructionsfor performing the acts or events of the various methods of the claimedsubject matter.

The aforementioned systems/circuits/modules have been described withrespect to interaction between several components/blocks. It can beappreciated that such systems/circuits and components/blocks can includethose components or specified sub-components, some of the specifiedcomponents or sub-components, or additional components, and according tovarious permutations and combinations of the foregoing. Sub-componentscan also be implemented as components communicatively coupled to othercomponents rather than included within parent components (hierarchical).Additionally, it should be noted that one or more components may becombined into a single component providing aggregate functionality ordivided into several separate sub-components, and any one or more middlelayers, such as a management layer, may be provided to communicativelycouple to such sub-components in order to provide integratedfunctionality. Any components described herein may also interact withone or more other components not specifically described herein but knownby those of skill in the art.

In addition, while a particular feature of the subject innovation mayhave been disclosed with respect to only one of several implementations,such feature may be combined with one or more other features of theother implementations as may be desired and advantageous for any givenor particular application. Furthermore, to the extent that the terms“includes,” “including,” “has,” “contains,” variants thereof, and othersimilar words are used in either the detailed description or the claims,these terms are intended to be inclusive in a manner similar to the term“comprising” as an open transition word without precluding anyadditional or other elements.

As used in this application, the terms “component,” “module,” “system,”or the like are generally intended to refer to a computer-relatedentity, either hardware (e.g., a circuit), a combination of hardware andsoftware, software, or an entity related to an operational machine withone or more specific functionalities. For example, a component may be,but is not limited to being, a process running on a processor (e.g.,digital signal processor), a processor, an object, an executable, athread of execution, a program, or a computer. By way of illustration,both an application running on a controller and the controller can be acomponent. One or more components may reside within a process or threadof execution and a component may be localized on one computer ordistributed between two or more computers. Further, a “device” can comein the form of specially designed hardware; generalized hardware madespecialized by the execution of software thereon that enables thehardware to perform specific functions; software stored on a computerreadable medium; or a combination thereof.

Moreover, the words “example” or “exemplary” are used herein to meanserving as an example, instance, or illustration. Any aspect or designdescribed herein as “exemplary” is not necessarily to be construed aspreferred or advantageous over other aspects or designs. Rather, use ofthe words “example” or “exemplary” is intended to present concepts in aconcrete fashion. As used in this application, the term “or” is intendedto mean an inclusive “or” rather than an exclusive “or”. That is, unlessspecified otherwise, or clear from context, “X employs A or B” isintended to mean any of the natural inclusive permutations. That is, ifX employs A; X employs B; or X employs both A and B, then “X employs Aor B” is satisfied under any of the foregoing instances. In addition,the articles “a” and “an” as used in this application and the appendedclaims should generally be construed to mean “one or more” unlessspecified otherwise or clear from context to be directed to a singularform. In another aspect, unless specified otherwise, or clear fromcontext, “transitioning between A and B” is intended to mean inclusivepermutations. That is transitioning from A to B and transitioning from Bto A.

Computing devices typically include a variety of media, which caninclude computer-readable storage media or communications media, inwhich these two terms are used herein differently from one another asfollows. Computer-readable storage media can be any available storagemedia that can be accessed by the computer, is typically of anon-transitory nature, and can include both volatile and nonvolatilemedia, removable and non-removable media. By way of example, and notlimitation, computer-readable storage media can be implemented inconnection with any method or technology for storage of information suchas computer-readable instructions, program modules, structured data, orunstructured data. Computer-readable storage media can include, but arenot limited to, RAM, ROM, EEPROM, flash memory or other memorytechnology, CD ROM, digital versatile disk (DVD) or other optical diskstorage, magnetic cassettes, magnetic tape, magnetic disk storage orother magnetic storage devices, or other tangible or non-transitorymedia which can be used to store desired information. Computer-readablestorage media can be accessed by one or more local or remote computingdevices, e.g., via access requests, queries or other data retrievalprotocols, for a variety of operations with respect to the informationstored by the medium.

On the other hand, communications media typically embodycomputer-readable instructions, data structures, program modules orother structured or unstructured data in a data signal that can betransitory such as a modulated data signal, e.g., a carrier wave orother transport mechanism, and includes any information delivery ortransport media. The term “modulated data signal” or signals refers to asignal that has one or more of its characteristics set or changed insuch a manner as to encode information in one or more signals. By way ofexample, and not limitation, communication media include wired media,such as a wired network or direct-wired connection, and wireless mediasuch as acoustic, RF, infrared and other wireless media.

What is claimed is:
 1. A system, comprising: a memory that storescomputer executable components; and a processor that executes at leastthe following computer executable components stored in the memory: animagery component configured to receive input indicating a viewpointfrom which to view a three-dimensional model of an object orenvironment; and a selection component configured to determine, based onthe input, a position of the viewpoint relative to the three-dimensionalmodel, and select a two-dimensional image from a plurality oftwo-dimensional images captured of the object or environment fordisplaying based in part on the two-dimensional image having a firstcapture position relative to the three-dimensional model that isassociated with a first navigational distance that is shorter thansecond navigational distances associated with second capture positionsof other two-dimensional images of the plurality of two-dimensionalimages, wherein the first navigational distance reflects one or morefirst constraints associated with navigation of the three-dimensionalmodel from the position to the first capture position, and wherein thesecond navigational distances reflect one or more second constraintsassociated with navigation of the three-dimensional model from theposition to the second capture positions.
 2. The system of claim 1,wherein the selection component is further configured to select thetwo-dimensional image based on the first navigational distance beingless than a threshold distance.
 3. The system of claim 1, wherein theviewpoint comprises a new viewpoint and wherein the computer executablecomponents further comprise: a construction component configured togenerate and display a representation of the three-dimensional modelfrom a previous viewpoint of the three-dimensional model, wherein theimagery component is configured to receive the input in association withthe display of the representation from the previous viewpoint; and atransition component configured to facilitate transitioning betweendisplaying the representation of the three-dimensional model todisplaying the two-dimensional image based on the input and selection ofthe two-dimensional image.
 4. The system of claim 3, wherein therepresentation comprises a three-dimensional reconstruction of thethree-dimensional model, and wherein the transition component isconfigured to replace the representation with the two-dimensional imageor a portion of the two-dimensional image data based on the input andselection of the two-dimensional image.
 5. The system of claim 3,wherein the representation comprises another two-dimensional image ofthe plurality of two-dimensional images, wherein the input comprisesselection of a point on the other two-dimensional image, and wherein thetransition component is configured to replace the representation withthe two-dimensional image or a portion of the two-dimensional image databased on the input and selection of the two-dimensional image.
 6. Thesystem of claim 1, wherein the viewpoint comprises a new viewpoint andwherein the input comprises a navigational command to move a virtualcamera relative to the three-dimensional model from a previous viewpointof the three-dimensional model to the new viewpoint, and wherein theselection component is further configured to determine the position ofthe new viewpoint relative to the three-dimensional model based on thecommand, the previous viewpoint, and a navigation mode employed to viewthe three-dimensional model.
 7. The system of claim 1, wherein the inputcomprises selection of a point on or within another two-dimensionalimage of the plurality of two-dimensional images, and wherein theselection component is further configured to determine the firstposition of the viewpoint relative to the three-dimensional model basedon the point.
 8. The system of claim 1, wherein the selection componentis further configured to determine a field of view for viewing the threedimensional model from the viewpoint, and select the two-dimensionalimage based on a difference between the field of view and a capturefield of view associated with the two-dimensional image being less thandifferences between the field of view associated with the viewpoint andcapture fields of view associated with other two-dimensional images ofthe plurality of two-dimensional images.
 9. The system of claim 1,wherein the position is associated with a room of the three-dimensionalmodel and wherein the selection component is further configured toselect the two-dimensional image based on the capture position of thetwo-dimensional image being associated with the room.
 10. The system ofclaim 1, wherein the one or more first or second constraints comprise abarrier constraint that restricts passage through solid objects inassociation with the navigation of the three-dimensional model.
 11. Thesystem of claim 1, wherein the one or more first or second constraintscomprise a room constraint that restricts the navigation of thethree-dimensional model to be within the same room.
 12. The system ofclaim 1, wherein the selection component is further configured to:determine an area of the three-dimensional model for viewing based onthe viewpoint; determine image data of the three-dimensional modelcorresponding to the area; and select the two-dimensional image based ona degree of correspondence between the image data and thetwo-dimensional image.
 13. The system of claim 1, wherein the viewpointcomprises a new viewpoint and wherein the selection component is furtherconfigured to select the two-dimensional image based on a previousviewpoint from which the three-dimensional model is viewed at a timewhen the input is received.
 14. The system of claim 13, wherein theinput comprises a navigational command that directs movement of avirtual camera away from the current viewpoint in association withnavigation of the three-dimensional model in a three-dimensionalwalkthrough mode.
 15. The system of claim 1, wherein the viewpointcomprises a new viewpoint and wherein the selection component isconfigured to determine the position of the new viewpoint by casting aray from a current viewpoint into the three-dimensional model in adirection determined based on the input, wherein the three-dimensionalmodel is viewed from the current viewpoint at a time when the input isreceived.
 16. A method, comprising using a processor to execute thefollowing computer executable instructions stored in a memory to performthe following acts: receiving input indicating a viewpoint from which toview a three-dimensional model of an object or environment; determining,based on the input, a position of the viewpoint relative to thethree-dimensional model; and selecting a two-dimensional image from aset of two-dimensional images captured of the object or environment fordisplaying based in part on the two-dimensional image having a firstcapture position relative to the three-dimensional model that isassociated with a first navigational distance that is shorter thansecond navigational distances associated with second capture positionsof other two-dimensional images of the set of two-dimensional images,wherein the first navigational distance reflects one or more firstconstraints associated with navigation of the three-dimensional modelfrom the position to the first capture position, and wherein the secondnavigational distances reflect one or more second constraints associatedwith navigation of the three-dimensional model from the position to thesecond capture positions.
 17. The method of claim 16, wherein theviewpoint comprises a new viewpoint and wherein the acts furthercomprise: displaying a representation of the three-dimensional modelfrom a previous viewpoint, wherein the receiving the input comprisesreceiving the input in association with interaction with therepresentation as displayed from the previous viewpoint; andtransitioning to displaying the two-dimensional image instead of therepresentation based on the selecting the two-dimensional image.
 18. Themethod of claim 17, wherein the representation comprises athree-dimensional reconstruction of the three-dimensional model.
 19. Themethod of claim 17, wherein the representation comprises anothertwo-dimensional image, and wherein the input comprises selection of apoint on the other two-dimensional image.
 20. The method of claim 16,wherein the receiving the input comprises receiving a command to move avirtual camera relative to a three-dimensional reconstruction of thethree-dimensional model from a previous viewpoint of thethree-dimensional model in association with navigation of thethree-dimensional model, and wherein the determining the positioncomprises determining the position based on the previous viewpoint. 21.The method of claim 16, wherein the acts further comprise: determining afield of view for viewing the three-dimensional model from theviewpoint, wherein the selecting the two-dimensional image furthercomprises selecting the two-dimensional image based on a differencebetween the field of view and a capture field of view associated withthe two-dimensional image being less than differences between the fieldof view associated with the viewpoint and capture fields of viewassociated with at least one other two-dimensional images of theplurality of two-dimensional images.
 22. The method of claim 16, whereinthe one or more first or second constraints comprise a barrierconstraint that restricts passage through solid objects in associationwith the navigation of the three-dimensional model.
 23. A system,comprising: a memory that stores computer executable components; and aprocessor that executes at least the following computer executablecomponents stored in the memory: an imagery component configured toreceive input indicating a location of a three-dimensional model of anobject or environment for viewing, wherein the input comprises selectionof a region on a displayed representation of the three-dimensionalmodel; and a selection component configured to: determine an area and adesired volume of the three-dimensional model for viewing based on theregion; and select a two-dimensional image from a plurality oftwo-dimensional images captured of the object or environment fordisplaying based in part on a score determined for the two-dimensionalimage that reflects a portion of the area and the volume of thethree-dimensional model that is visible in the two-dimensional image.24. The system of claim 23, wherein the displayed representation of thethree-dimensional model was rendered from a current viewpoint relativeto the three-dimensional model, wherein the selection component isconfigured to determine a distance between the current viewpoint and theregion, and wherein the selection component is further configured todetermine the area based on the distance.
 25. The system of claim 23,wherein the selection component is further configured to: determineimage data of the three-dimensional model corresponding to the area; andselect the two-dimensional image based on the score being determined forthe two-dimensional image based on a degree of correspondence betweenthe image data and the two-dimensional image.
 26. The system of claim25, wherein the image data comprises an image of the three-dimensionalmodel from a viewpoint relative to the three-dimensional modeldetermined based in part on the location.
 27. The system of claim 23,wherein the selection component is configured to determine a centerpoint of the desired volume of the three-dimensional model for viewingbased on an average position of points included in the selected regionrelative to a current viewpoint of the three-dimensional model, andwherein the selection component is further configured to determine thearea based on the center point.
 28. The system of claim 23, wherein theselection component is configured to determine a desired field of viewfor viewing the three-dimensional model based on a standard deviation ofpositions of points included in the selected region relative to acurrent viewpoint of the three-dimensional model, and wherein theselection component is further configured to determine the area based onthe desired field of view.
 29. A method comprising: receiving, by asystem comprising a processor, input selecting a region of a displayedrepresentation of a three-dimensional model of an object or environmentfor viewing from a new perspective; determining, by the system, a volumeof the three-dimensional model based on the region; and selecting, bythe system, a two-dimensional image from a plurality of two-dimensionalimages captured of the object or environment for displaying based inpart on a score determined for the two-dimensional image that reflectsan amount of the volume of the three-dimensional model that is visiblein the two-dimensional image.
 30. The method of claim 29, wherein thedisplayed representation of the three-dimensional model reflects acurrent viewpoint relative to the three-dimensional model, and whereinthe determining the volume is further based on a distance between thecurrent viewpoint and the region.
 31. The method of claim 29, furthercomprising: determining, by the system, image data of thethree-dimensional model corresponding to the volume, and wherein theselecting the two-dimensional image is further based on the score beingdetermined based on a degree of correspondence between the image dataand the two-dimensional image.
 32. The method of claim 31, wherein theimage data comprises an image of the three-dimensional model from aviewpoint relative to the three-dimensional model determined based inpart on the region.
 33. The method of claim 29, further comprising:determining, by the system, a center point for the volume of thethree-dimensional model based on an average position of points includedin the region relative to a current viewpoint of the three-dimensionalmodel, and wherein the determining the volume is further based on thecenter point.
 34. The method of claim 29, further comprising:determining, by the system, a field of view for viewing thethree-dimensional model based on a standard deviation of positions ofpoints included in the region relative to a current viewpoint of thethree-dimensional model, and wherein the determining the volume isfurther based on the field of view.